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SUMMARY 


While  the  objective  of  every  aircrew  is  to  accomplish  their  mission,  every  effort  must  be  made  to 
ensure  the  safety  of  the  crew.  Air  Force  Air  Mobility  Command  (AMC)  has  been  flying  more 
and  longer  missions  with  fewer  pilots  and  fatigue  has  become  a  potential  safety  issue.  Safety  is 
compromised  when  aircrews  are  perfonning  their  mission  tasks  in  a  fatigued  state.  A  fatigue 
assessment  and  management  tool  built  on  a  scientifically  based  model  of  sleep  and  cognitive 
perfonnance  can  be  helpful  in  managing  the  fatigue  problem.  Minor  changes  to  a  schedule  may 
eliminate  or  mitigate  crew  fatigue.  The  purpose  of  this  report  is  to  document  a  portion  of  the 
work  conducted  under  a  Phase  3  SBIR  contract  to  develop  a  web-based,  fatigue  management 
tool  with  interfaces  for  diverse  user  applications.  The  Fatigue -Perfonnance  Assessment  System 
(F-PAS)  was  derived  from  the  Fatigue,  Avoidance  Scheduling  Tool  (FAST™),  which  contains 
the  Sleep,  Activity,  Fatigue,  and  Task  Effectiveness  (SAFTE™)  model  that  forecasts  cognitive 
perfonnance  level  based  on  sleep,  circadian  rhythm,  and  sleep  inertia.  Specifically,  this  report 
describes  the  Mission  Scheduler  Interface  of  the  24/7  Operational  Toolset  to  be  used  by  AMC 
schedulers,  pilots,  and  flight  surgeons.  The  interface  was  designed  to  aid  in  the  scheduling  of 
military  missions  and  duty  periods  such  that  the  effects  of  mental  fatigue  on  human  performance 
are  minimized.  Once  sleep  times  are  detennined  the  tool  projects  performance  effectiveness  for 
critical  points  within  a  mission.  Aircrew  and  flight  surgeons  can  use  the  tool  to  evaluate 
phannaceutical  fatigue  countenneasures  when  sufficient  sleep  cannot  be  scheduled  for  a  critical 
long  duration  or  nighttime  mission.  Aircrew  can  also  use  the  tool  to  evaluate  the  fatigue  impact 
of  changes  to  an  interrupted  mission  schedule.  The  investigators’  used  a  task-centered,  system 
design  involving  task  analysis  to  develop  the  requirements  for  the  interface.  The  interface  design 
approach  was  iterative,  involving  meetings  among  subject  matter  experts  from  each  of  the  three 
user  groups,  interface  software  designers,  and  evaluators.  The  report  includes  task  descriptions, 
an  interface  description,  and  usability  results,  which  revealed  that  the  early  Beta  implementation 
of  F-PAS  was  effective  for  detennining  fatigue  effects  on  perfonnance.  Incomplete  or 
unfinished  features  of  the  software  were  not  tested. 
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INTRODUCTION 


This  report  describes  the  Mission  Scheduler  Interface  of  the  24/7  Operational  Toolset.  The 
toolset  was  based  upon  the  Sleep,  Activity,  Fatigue,  and  Task  Effectiveness  (SAFTE™;  Hursh  et 
ah,  2004).  The  SAFTE™  model  predicts  cognitive  perfonnance  level  based  upon  sleep, 
circadian  rhythm,  and  sleep  inertia.  This  specific  interface  of  the  toolset  was  designed  to  aid  in 
the  scheduling  of  military  missions  and  duty  periods  such  that  the  effects  of  mental  fatigue  on 
human  performance  are  minimized.  The  interface  design  approach  was  iterative,  involving 
several  meetings  among  subject  matter  experts  (SMEs),  interface  software  designers  and 
evaluators.  Research  revealed  that  Air  Mobility  Command  (AMC)  would  likely  be  the  primary 
user  of  a  fatigue  management  product  because  of  the  multi-day  missions  they  routinely  fly. 

In  our  interactions  with  potential  users,  the  first  meeting  was  for  the  purpose  of  requirements 
analysis,  in  which  the  investigators  elicited  task  information  from  the  SMEs.  Three  user  groups 
were  interviewed,  schedulers,  pilots,  and  flight  surgeons.  The  second  meeting  included  a  walk¬ 
through  of  storyboarded  and  preliminary  software,  in  which  the  SMEs  provided  feedback  to  the 
investigators  and  evaluators.  The  final  meeting  was  for  the  purpose  of  an  “inspection 
evaluation”  of  the  interface  by  SMEs  and  evaluators.  The  report  begins  with  a  general  review  of 
scheduling  procedures  for  mobility  aircraft,  the  primary  context  within  which  the  interface  may 
be  used.  The  task  analysis  of  the  scheduler’s  tasks  is  followed  by  comments  from  users  of  the 
PC-based  product,  the  Fatigue  Avoidance  scheduling  Tool  (FAST™).  Following  a  description 
of  the  methods  used  in  the  task  analysis,  the  results  are  presented  including  usability  testing.  The 
report  ends  with  recommendations  and  suggestions  for  incorporating  a  fatigue-assessment 
decision  aid  into  a  mission  scheduling  process. 

SCHEDULING  PROCEDURES  FOR  MOBILITY  AIRCRAFT 

While  the  objective  of  every  aircrew  is  to  accomplish  their  mission,  every  effort  must  be  made  to 
ensure  the  safety  of  the  crew.  Safety  is  compromised  when  aircrews  are  perfonning  their 
mission  tasks  in  a  fatigued  state.  Minor  changes  to  a  schedule  may  eliminate  or  mitigate  crew 
fatigue. 

The  Air  Mobility  Command  (AMC)  was  created  June  1,  1992.  AMC  provides  America's  ability 
to  reach  around  the  world.  This  rapid,  flexible  and  responsive  air  mobility  promotes  stability  in 
regions  by  keeping  America's  capability  and  character  highly  visible. 

Air  Mobility  Command's  mission  is  to  deliver  maximum  war-fighting  and  humanitarian  effects 
for  America  through  rapid  and  precise  global  air  mobility.  The  command  plays  a  crucial  role  in 
providing  humanitarian  support  at  home  and  around  the  world.  AMC  Airmen— active  duty,  Air 
National  Guard,  Air  Force  Reserve,  and  civilians— provide  airlift  and  aerial  refueling  for  all  of 
America's  anned  forces.  Many  special  duty  and  operational  support  aircraft  and  stateside 
aeromedical  evacuation  missions  are  also  assigned  to  AMC. 

United  States  (US)  military  forces  must  be  able  to  provide  a  rapid,  tailored  response  with  a 
capability  to  intervene  against  a  well-equipped  foe,  hit  hard,  and  terminate  quickly.  Rapid  global 
mobility  lies  at  the  heart  of  US  strategy  in  this  environment— without  the  capability  to  project 
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forces,  there  is  no  conventional  deterrent.  As  US  forces  stationed  overseas  continue  to  decline  in 
numbers,  global  interests  remain,  keeping  the  unique  capabilities  AMC  can  provide  in  demand. 

As  the  air  component  of  the  U.S.  Transportation  Command,  AMC  serves  many  customers  and, 
as  the  single  manager  for  air  mobility,  AMC's  customers  have  only  one  number  to  call  for  Global 
Reach.  Air  lifters  provide  the  capability  to  deploy  armed  forces  anywhere  in  the  world  and  help 
sustain  them  in  a  conflict.  Air  refueling  aircraft  are  the  lifeline  of  Global  Reach.  Since  Air 
Force  tankers  can  also  refuel  Navy,  Marine  and  many  allied  aircraft,  they  leverage  all  service 
capabilities  on  land,  sea  and  in  the  air.  These  aircraft  also  have  an  inherent  cargo-carrying 
capability— maximizing  AMC's  lift  options. 

AMC's  mission  encompasses  nearly  140,000  active-duty  and  Air  Reserve  Component  military 
and  civilian  personnel.  They  include  approximately  46,700  active  duty,  8,300  civilians,  46,600 
Air  Force  Reserve  and  38,300  Air  National  Guard. 

AMC's  provides  mobility  aircraft  such  as  the  C-5  Galaxy,  KC,  C-130  Hercules,  and  KC-135 
Stratotanker  (see  Figure  1)  and  operational  support  aircraft  such  as  the  C-9,  C-20,  and  UH-1. 


Figure  1.  A  KC-135  Stratotanker  from  the  100th  Air  Refueling 
Wing,  Mildenhall,  England  (U.S.  Air  Force  photo/Master  Sgt.  Mark 
Bucher). 


AMC  has  one  numbered  air  force,  the  18th  Air  Force,  which  is  charged  with  tasking  and 
executing  all  air  mobility  missions1 2.  Units  reporting  to  18th  Air  Force  include  all  Air  Mobility 
Command  wings  and  groups  based  in  the  continental  United  States,  as  well  as  two  expeditionary 
mobility  task  forces— the  15th  EMTF  and  the  21st  EMTF.  The  15th  and  21st  EMTFs  serve  as 


1  http://www.amc.af.mil/library/factsheets/factsheet.asp7kU231,  current  as  of  June,  2007. 
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lead  agencies  for  conducting  mobility  operations  worldwide.  They  are  critical  to  the  execution 
phase  of  war  fighting  by  providing  worldwide  expeditionary  mobility  support.  The  618th  Tanker 
Airlift  Control  Center  (TACC),  also  reports  to  18th  Air  Force  and  serves  as  the  organization’s  air 
operations  hub,  planning  and  directing  tanker  and  transport  aircraft  operations  around  the  world. 
AMC  also  has  a  number  of  active-duty  bases  and  one  major  direct  reporting  unit,  the  USAF 
Expeditionary  Center  which  serves  as  the  Air  Force's  premier  organization  for  expeditionary 
innovation,  education,  training  and  exercises. 

A  new  era  in  air  power  history  began  June  1,  1992  when  the  Military  Airlift  Command  and  the 
Strategic  Air  Command  were  inactivated  and  Air  Mobility  Command  formed  from  elements  of 
these  two  organizations.  AMC  melded  a  worldwide  airlift  system  with  a  tanker  force  that  had 
been  freed  from  its  commitments  by  the  collapse  of  the  Soviet  Union.  AMC  has  undergone 
considerable  change  since  its  establishment.  Focusing  on  the  core  mission  of  strategic  air 
mobility,  the  command  divested  itself  of  infrastructure  and  forces  not  directly  related  to  Global 
Reach.  On  Oct.  1,  2003,  AMC  underwent  a  major  restructuring,  bringing  a  war  fighting  role  to 
its  numbered  air  force.  AMC  reactivated  18th  Air  Force  and  re-designated  its  two  fonner 
numbered  air  forces  as  the  15th  EMTF,  and  the  21st  EMTF. 

The  AMC  Fact  Sheet  for  2007  provides  the  following  infonnation  on  AMC's  ability  to  provide 
global  reach  on  a  daily  basis.  From  providing  fuel,  supplies  and  aeromedical  support  to  troops 
on  the  frontline  of  the  Global  War  on  Terrorism,  to  providing  humanitarian  supplies  to  hurricane, 
flood,  and  earthquake  victims  both  at  home  and  abroad,  AMC  has  been  engaged  in  almost 
nonstop  operations  since  its  inception.  Command  tankers  and  airlifters  have  supported 
peacekeeping  and  humanitarian  efforts  in  Afghanistan,  Bosnia,  Iraq,  Cambodia,  Somalia, 

Rwanda  and  Haiti,  and  continue  to  play  a  vital  role  in  the  ongoing  Global  War  on  Terrorism. 
These  many  examples  of  the  effective  application  of  non-lethal  air  power  indicate  that  air 
mobility  is  a  national  asset  of  growing  importance  for  responding  to  emergencies  and  protecting 
national  interests  around  the  globe. 
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Overall  Mission  Scheduling  Processes 


AMC  schedulers  create  itineraries  for  their  aircraft  and  crews  to  accomplish  missions  required 
for  combat  operations,  base  needs,  the  Pentagon,  other  DoD  authorities,  or  civilian  emergencies. 
Reserve  and  National  Guard  units  create  schedules  for  TACC  and  for  their  own  operations 
locally.  The  tools  they  use  for  their  own  schedules  are  scattered  about  in  manuals,  books, 
pamphlets,  on  the  World  Wide  Web,  and  in  their  heads  from  training  and  experience.  However, 
they  also  select  crews  and  aircraft  to  accomplish  missions  mandated  by  TACC,  who  plans  the 
basic  mission. 

AMC  schedulers  in  the  regular  AF  generally  are  located  at  Scott  AFB  and  use  the  Global 
Decision  Support  System  (GDSS).  This  Web-based  system  is  AMC’s  force-level  Command  and 
Control  (C  )  system  supporting  AMC/TACC  execution  authority  for  effective  airlift  mission 
management.  GDSS  consists  of  nodes  located  at  different  locations  that  continuously  replicate 
information  to  keep  each  node  updated  with  the  latest  information.  The  objective  of  the  GDSS 
program  is  to  improve  AMC’s  C2  force-level  decision  making  by  providing  its  users  with 
automated  capabilities  to  support  airlift  planning  and  execution,  aircraft  schedule  dissemination, 
aircrew  management,  and  mission  management  of  AMC’s  airlift  and  air  refueling  missions.  Its 
purpose  is  to  provide  a  fully  functional,  operational  system  that  satisfies  the  C  support 
requirements  of  AMC.  GDSS  interfaces  with  several  C2  systems,  including  Command  and 
Control  Information  Processing  System  (CGPS),  the  wing-level  C  planning  and  execution 
system,  AMC  Deployment  Analysis  System  (ADANS),  and  the  USTRANSCOM  Global 
Transportation  Network  (GTN). 

The  following  information  was  taken  from  Air  National  Guard  Instruction  10-207,  2004,  Chapter 
3,  page  8.  All  AF  units  rely  heavily  on  GDSS  to  maintain  visibility  of  their  airlift,  tanker  and 
Strat.  missions.  A  large  global  USAF  audience  views  GDSS  and  other  AF  software  ties  into  and 
works  off  GDSS.  The  Command  and  Control  Infonnation  Processing  System  (C2IPS)  is  a  unit 
level  C"  system  that  manages  functions  such  as  communications  processing,  message/data 
processing  and  display,  and  nodal  data  networking.  At  the  wing  level,  it  channels  infonnation 
between  the  air  transportation,  intelligence,  maintenance,  operations,  supply,  weather,  and 
surgeon  general  functions.  Unclassified  infonnation  is  passed  between  unclassified  GDSS  and 
C2IPS  Intelligent  Messaging  Units  at  Scott  AFB,  Illinois  and  Travis  AFB,  California. 
Unclassified  GDSS  passes  schedule  and  execution  data  to  C2IPS.  C2IPS  passes  anival  and 
departure  information  to  GDSS  and  the  next  three  down-line  stations.  For  missions  consisting  of 
more  than  three  legs,  GDSS  passes  that  infonnation  to  the  other  C2IPS  equipped  down-line 
stations.  C“IPS  transmits  takeoff,  landing,  diversion,  over-fly,  schedule,  diplomatic  clearance, 
and  Unit  Line  Number  (ULN)  information  (number  of  passengers  and  tons  of  cargo)  to 
unclassified  GDSS.  Units  using  C  IPS  input:  mission  departures,  arrivals,  deviations,  recuts, 
diverts,  overflys,  delay  codes,  and  advisories  into  the  system.  Unit  Commanders  develop 
procedures  to  assure  that  the  Operations  Center  is  provided  en  route  times  for  each  leg  until  a 
mission  is  tenninated.  It  is  a  unit’s  responsibility  to  close  their  mission. 

The  Airlift  Information  and  Reporting  System  (AIRS)  was  designed  primarily  to  assist  the  unit 
scheduler  in  building  missions  and  trips  and,  upon  trip  completion,  producing  after  action 
reports.  The  AIRS  was  not  designed  to  fully  support  C2  flight  following,  however.  Units  must 
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load  their  itineraries  14  days  or  more  before  departure  for  the  system  to  function  properly.  At 
each  departure  time  for  airlift  missions,  the  total  passenger  and/or  cargo  on  board  must  be  loaded 
into  C"IPS  or  AIRS.  Times  for  transitory  aircraft  arriving  or  departing  a  location  will  be  either 
updated  via  C2IPS  or  reported  to  the  Operations  Center  (Air  National  Guard  Instruction  10-207, 
2004). 

Typical  Last  Minute  Mission  Schedule 

Geoff  Janes  of  the  100th  Air  Refueling  Wing  Public  Affairs  wrote  a  story  published  in  the  Air 
Force  News  about  a  last  minute  AMC  refueling  operation  that  is  typical  of  rushed  scheduling2. 

A  synopsis  of  that  story  follows. 

A  late  flight  call  and  a  cancelled  sortie  led  an  aircrew  from  the  351st  Air  Refueling  Squadron  at 
Meldenhall,  England  to  expedite  medical  care  for  more  than  a  dozen  severely  injured  troops  being 
transported  from  Iraq  to  Andrews  Air  Force  Base,  Maryland,  on  Feb.  7,  2007.  According  to  the 
100th  Operations  Support  Squadron  scheduler,  the  refueling  mission  was  far  from  the  norm.  They 
got  a  call  around  2:30  a.m.  asking  if  they  could  refuel  a  high-priority  air-evacuation  mission  (en 
route)  to  the  hospital  at  Andrews.  Luckily  they  had  a  cancelled  flight  and  a  crew  available.  That 
captain  and  crew  had  been  scheduled  to  fly  his  first  mission  as  aircraft  commander  on  a  routine 
refueling  mission  over  the  Mediterranean  Sea.  The  pilot  said  that  they  normally  know  24  to  48 
hours  prior  to  a  flight.  However,  when  they  showed  up,  their  (mission)  binder  (still)  had  all  the 
information  from  the  previous  (cancelled)  flight.  The  aircraft  commander  said  what  information 
they  did  have  on  the  new  mission  was  the  refueling  track,  the  time  of  the  rendezvous  and  the  call 
sign  of  the  receiver,  a  C-17  Globemaster  III  from  the  Mississippi  Air  National  Guard  that  had  left 
Iraq  at  about  1  a.m.  Greenwich  Mean  Time  ( GMT).  The  Air  Guard's  mission  was  unique  as  the 
majority  of  the  flights  are  to  the  Landstuhl  Regional  Medical  Facility  in  Germany.  A  Mississippi 
Air  National  Guardsman  spokesperson  said  that  their  crews  are  able  to  make  changes  to  meet  the 
needs  of  the  Air  Force  as  the  mission  dictates.  However,  there  was  a  tremendous  amount  of  work 
they  had  to  do  to  make  this  mission  happen. 

The  same  can  be  said  of  the  crews  at  Mildenhall.  The  maintainers  and  aircrew  worked 
unbelievably  fast  because  they  realized  how  critical  the  mission  was.  They  basically  planned  it  all 
from  scratch.  The  KC-135  launched  from  RAF  Mildenhall  at  6:30  a.m.,  and  passed  more  than 
16,000  gallons  of  fuel  to  the  C-17  over  the  England-Scotland  border.  The  C-17  commander  said 
that  they  arrived  in  Maryland  just  before  3  p.m.  GMT.  Figure  1  shows  a  KC-135  Stratotanker 
from  the  100th  Air  Refueling  Wing,  taking  off  from  Royal  Air  Force  Mildenhall  on  Feb.  7,  2007  on 
an  unscheduled  mission  to  refuel  a  C-17  Globemaster  III  that  was  rushing  injured  troops  from 
Iraq  to  Andrews  Air  Force  Base,  Maryland.  The  rendezvous  saved  the  air-evacuation  mission 
some  three  hours  of  flying  time. 

On  the  trip  back  to  RAF  Mildenhall,  the  crew  ran  into  a  snow  storm  that  required  them  to  circle 
the  base  before  landing  on  a  runway  that  had  just  been  cleared  by  a  snow  plow.  The  commander 
said  the  refueling  mission  saved  the  C-17  crew  roughly  three  hours  it  would  have  taken  for  them 
to  land  and  refuel.  He  said,  "We  weren't  the  ones  carrying  (the  injured  troops),  but  who  knows? 

We  might  have  saved  them  a  few  hours  that  made  the  difference  between  life  and  death.  But  then  I 
thought  to  myself  after  we  landed  that  I  get  to  go  home  today  while  the  guys  in  the  back  of  that 
plane  are  fighting  for  their  lives.  It  was  sobering.  " 

This  story  of  a  hurry-up  mission  is  not  atypical  of  missions  scheduled  by  AMC  schedulers.  In 
this  case,  they  were  blessed  with  a  rested  aircrew  and  an  available  plane  to  accomplish  the 


2  Taken  from  a  story  written  by  Geoff  Janes,  100th  Air  Refueling  Wing  Public  Affairs,  2/14/2007  -  Royal  Air  Force 
Mildenhall,  England  (AFNEWS). 
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mission.  However,  schedulers  do  not  have  any  knowledge  of  the  fatigue  impact  of  the  schedule 
they  create  for  the  crew  and  must  do  their  planning  based  on  a  set  of  rules  established  over  time 
that  merely  have  historical  precedence.  The  rules  do  not  take  into  account  the  sleep  the  crew  has 
received  or  their  circadian  rhythm,  only  that  they  have  been  given  an  opportunity  for  a  prescribed 
amount  of  “crew  rest,”  usually  12  hours.  A  model  that  takes  into  account  the  time  and  duration 
of  the  crewmembers’  sleep  can  project  their  cognitive  perfonnance  effectiveness  for  the  mission 
and  the  return  flight  (Hursh  et  ah,  2004).  It  can  assess  performance  effectiveness  during  critical 
events  like  take-off,  landing,  and  air  refueling.  Once  a  scheduler  knows  that  perfonnance 
effectiveness  is  poor  for  the  critical  events  of  a  tentative  mission,  it  may  be  possible  to  either 
better  rest  the  crew  for  the  mission,  reschedule  the  events  to  a  time  when  the  crew  is  less 
fatigued,  or  implement  other  specific,  fatigue-risk-mitigating  operational  risk  management 
(ORM)  procedures. 
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METHODS 


A  more  detailed  explanation  of  the  methods  used  here  may  be  viewed  in  the  companion  technical 
report,  24/7  Operational  Effectiveness  Toolset:  Usability  Assurance  Plan  (Miller,  Eddy  &  Moise, 
2008). 

REQUIREMENTS  ANALYSIS 
AMC  Schedulers 

We  conducted  a  task  analysis  (TA)  on  three  AMC  Reservist  schedulers  to  help  us  incorporate  the 
fatigue  management  tool  into  the  scheduler’s  workflow.  We  used  a  modified  version  of  the  TA 
methods  described  by  Greenberg  (2004)  to  uncover  their  scheduling  goals,  processes,  and  tasks. 
In  addition,  we  reviewed  documents  recommended  by  the  schedulers  to  understand  their  work 
processes.  Subsequently,  we  used  the  data  from  the  TA  to  identify  where  best  to  apply  fatigue 
management  in  the  scheduling  process.  Because  we  were  aware  of  the  information  requirements 
of  the  fatigue  management  process,  we  were  able  to  identify  the  scheduler’s  tasks  from  which  the 
necessary  infonnation  could  be  acquired.  Once  we  extracted  the  necessary  data  from  the 
workflow  and  entered  into  the  fatigue  management  tool,  the  next  step  was  to  detennine  where  in 
the  workflow  the  fatigue  analysis  could  best  be  presented  to  the  scheduler.  The  TA  allowed  us  to 
identify  the  mission  scheduling  processes  that  could  benefit  from  this  information,  but  would 
minimally  impact  the  scheduler’s  other  tasks. 

Since  the  purpose  of  a  flight  schedule  is  to  accomplish  a  specific  mission,  fatigue  considerations 
of  the  schedule  have  always  been  secondary.  That  is,  the  flight  events  schedule  is  initially 
created  to  support  the  mission;  it  is  not  designed  to  avoid  or  minimize  aircrew  fatigue. 

Therefore,  we  must  assume  that,  typically,  fatigue  analysis  will  follow  event  scheduling  to 
support  the  mission.  Given  those  priorities,  the  scheduler  will  need  to  conduct  “What-If  ’ 
analyses  on  the  schedule,  trying  various  modifications  to  mitigate  fatigue,  but  being  sure  to  allow 
effective  and  efficient  mission  accomplishment.  Therefore,  considerable  effort  was  expended  by 
all  participants  to  identify  how  a  scheduler  would  edit  a  schedule  with  minimal  effort  while 
maintaining  the  “Big  Picture”  of  the  overall  mission  and  its  objective(s). 

We  taught  the  subject  matter  expert  (SME)  schedulers  how  to  use  the  Fatigue  Avoidance 
Scheduling  Tool  (FAST™)  to  help  them  understand  what  the  fatigue  management  tool  required 
as  input  and  what  it  could  produce  as  output.  We  used  the  SBIR  prototype  product  currently  in 
use  within  the  AF,  Navy,  and  select  commercial  sites  (Eddy  &  Hursh,  2001,  2006a,  2006b).  The 
FAST™  software  was  introduced  after  the  TA  was  conducted.  This  experience  with  FAST™ 
allowed  the  SMEs  to  see  how  and  where  its  data  and  processes  could  fit  into  their  scheduling 
processes  and  tasks. 

A  modified,  Goal-Directed,  Task-Centered  System  Design  Approach  was  used  fashioned  after 
Greenberg’s  (2004)  4-step  version  of  the  method  by  Fewis  &  Rieman.  After  eliciting  this 
information,  the  SMEs  were  trained  on  FAST™  so  that  they  could  see  what  information  a 
fatigue  management  tool  would  need  and  what  types  of  displays  it  could  produce.  They  then 
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inserted  the  data  acquisition  and  output  requirements  for  the  tool  into  their  scheduling  tasks  at 
the  appropriate  places. 

Three  Air  Force  Reserve  Command  (AFRC)  schedulers  sponsored  by  the  HQ  of  the  22nd  AF 
(AFRC)  served  as  subject  matter  experts  (SMEs)  for  the  TA.  Three  different  wings,  the  3 15th 
AW,  Charleston,  AFB  (AFRC),  the  5 14th  AMW,  McGuire  AFB  (AFRC),  and  the  436th  AW, 
Dover  AFB  (AMC)  supported  the  effort,  each  sending  a  scheduler. 

The  three  SMEs  and  five  investigators/developers  conducted  the  TA  and  all  discussions  in  a 
small  conference  room  at  Brooks  City-Base,  Texas.  At  one  end  of  the  room  a  large  projection 
screen  was  used  to  record  the  scheduler’s  processes  and  tasks  from  the  structured  TA.  One  of 
the  four  investigators  led  the  TA;  one  recorded  the  infonnation  on  a  laptop  computer  in  a 
Word™  table  projected  onto  the  screen.  The  other  three  investigators  prompted  the  SMEs  for 
additional  infonnation  and  took  supplementary  notes  on  the  discussions.  Three  of  the 
investigators  were  pilots  and  one  was  a  reserve  safety  officer.  Two  investigators  were  fatigue 
management  knowledgeable.  Three  of  the  investigators  were  fully  familiar  with  FAST™. 

After  all  of  the  SMEs  had  introduced  themselves,  the  investigators  stated  the  goals,  objectives, 
and  agenda  for  the  meeting  and  answered  questions.  After  introducing  the  TA  procedures,  the 
process  began.  The  process  started  with  a  brief  introduction  to  fatigue  and  its  management  using 
graphs  to  show  how  knowing  about  sleep  and  circadian  rhythm  provided  critical  infonnation  that 
predicted  performance  capability.  Concepts  from  the  Sleep,  Activity,  Fatigue,  and  Task 
Effectiveness  (SAFTE™)  model  and  FAST™  graphs  (Eddy  &  Hursh,  2001,  2006b)  illustrated 
fatigue  and  performance  effects,  but  the  FAST™  interface  was  not  described  at  the  time.  We  did 
not  want  to  bias  schedulers  toward  the  current  FAST™  interface,  so  we  conducted  the  TA  before 
teaching  them  anything  about  FAST™. 

The  essence  of  the  TA  plan  was  to  have  the  SMEs  walk  through  planning  a  mission  describing 
the  processes  and  information  required  while  we  created  a  visual  recording  of  what  they  said.  So 
that  the  schedulers  could  see  what  we  were  recording  at  all  times  and  make  corrections  as 
necessary,  we  projected  our  recordings  in  a  Word™  Table  onto  a  large  screen.  This  approach 
helped  to  counter  the  limitations  of  our  working  memories  so  we  would  not  have  to  remember 
what  was  said  earlier  and  we  could  see  infonnation  in  the  context  of  previously  discussed  facts. 

We  met  with  the  three  schedulers  for  about  12  hours  over  one  and  a  half  days.  We  conducted  the 
TA  during  the  morning  of  the  first  day,  spent  the  afternoon  teaching  them  FAST™,  had  them 
review  their  inputs  to  the  TA  for  two  hours  the  second  morning,  and  then  conducted  an 
abbreviated  usability  analysis  on  FAST™  during  the  last  two  hours  of  the  second  morning. 

After  the  fatigue  management  introduction,  we  began  our  high-level,  user-centered  requirements 
analysis  by  having  the  schedulers  tell  us  which  types  of  users  to  include.  Who  are  the  target 
users  for  information  on  fatigue  that  could  use  it  to  save  lives,  make  operations  more  efficient, 
and  reduce  stress  on  the  aircrews?  We  had  them  tell  us  who  absolutely  must  be  included,  who 
should  be  included  if  possible,  and  who  should  be  excluded.  Once  we  defined  who  the  users  of 
our  product  would  be,  we  moved  on  to  define  what  their  skills  and  abilities  were  for  creating  a 
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mission  schedule.  We  also  asked  them  what  the  training  were  for  doing  their  mission-planning 
job.  After  this,  we  asked  about  how  they  did  their  job. 

We  started  the  TA  process  by  asking  them  to  conceptualize  a  specific  mission  that  they  may  have 
planned  recently.  Then  we  asked  them  to  back  up  to  the  beginning  and  tell  us  what  was  needed 
to  plan  such  a  mission.  We  ask  them  to  provide  us  with  the  top-level  processes  they  used  and 
goals  they  followed  as  they  assembled  a  mission.  The  processes  we  were  looking  for  started 
from  the  time  they  obtained  the  directive  to  do  a  mission,  to  when  that  mission/flight  was 
planned/accepted,  and  out  the  door.  As  they  provided  this  infonnation,  we  recorded  it  in  the  left 
column  of  a  three-column  table  projected  on  a  screen  in  front  of  them.  The  columns  were 
labeled:  Processes/Goals,  Tasks  to  Accomplish,  and  Suggested  Modifications  to  FAST™.  The 
first  column  was  completed  before  going  to  the  second  and  third  columns.  As  they  created  their 
schedule,  we  wanted  to  identify  their  goals  and  major  processes  before  they  described  the 
associated  individual  tasks  to  accomplish  them. 

Once  the  major  processes  and  goals  were  delineated,  we  asked  them  to  tell  us  all  the  tasks  that 
were  required  under  each  of  the  top-level  processes.  This  often  caused  them  to  describe 
additional  processes  and  goals  or  sub  processes  that  were  added  to  the  Processes/Goals  column. 
There  was  also  some  consolidation  and  reformulation  of  processes  and  goals  as  they  described 
the  tasks  under  the  various  processes.  We  told  them  that  as  we  move  through  the  design  process 
and  exercise,  we  are  essentially  trying  to,  like  an  onion,  “peel  away”  the  layers  of  a  mission 
planner’s  knowledge  and  experience  to  understand  the  key  points  of  “what”  and  “how”  they  do 
their  job.  Over  the  day  and  half  we  felt  as  though  we  learned  the  basics  of  what  we  needed  to 
begin  designing  an  interface  for  AMC  schedulers. 

On  the  second  day,  after  the  SMEs  had  received  training  on  FAST™,  we  asked  them  to  go  back 
to  the  TA  chart  and  correlate  FAST™  features  with  their  scheduling  processes  and  tasks.  The 
idea  was  to  see  what  FAST™  processes  met  their  schedule  building  task  needs  and  what  was 
lacking  in  the  current  tool.  We  also  had  them  critique  alternate  ways  of  perfonning  some  of  the 
FAST™  operations,  such  as  editing  schedules.  This  infonnation  was  recorded  in  column  3  of 
the  TA  and  Usability  Chart  projected  before  them.  Afterward  we  asked  them  to  establish 
priorities  for  the  design  constructs  indicating  those  that  they  absolutely  must  have,  those  to  do  if 
budget  and  time  permit,  and  those  that  are  just  nice  to  have.  Also  at  the  end,  all  SMEs  and 
investigators  were  asked  to  verbalize — and  write—  post-it  notes  and  other  notes  as  we  reviewed 
the  table  of  goals,  processes,  and  tasks  and  make  additional  suggestions.  This  was  to  enable  us 
to  “capture”  a  lot  of  information  quickly  as  it  came  to  mind. 

Caveats.  We  recognized  that  AFRC  and  AMC  might  have  different  needs,  so  our  plan  was  to 
use  AFRC  to  establish  common  ground  regarding  the  interface,  and  then  identify  exceptions  for 
AMC  mission  planners.  Our  underlying  assumptions  concerning  the  differences  between  AMC 
and  AFRC  were  as  follows: 

•  “AMC”  refers  to  the  overlying  goal  of  being  able  to  assess  AMC-directed  missions  for 
“effectiveness”  with  regard  to  fatigue  issues,  and  using  the  tool  as  a  planning  aid  to 
assess,  and  perhaps  mitigate  fatigue -related  problems  within  a  mission  and  safety  context. 

•  “AFRC”  refers  to  the  overlying  goal  of  using  the  tool  to  optimize  missions  during 
planning  for  the  Air  Force  Reserve  Command  components  from  a  fatigue  standpoint. 
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The  question  is  how  do  we  do  a  better  job  on  the  interface  given  ARFC  user  constraints 
to  use  the  tool  as  a  decision  aid?  This  means  the  additional  capability  to  assess  “what  if’ 
scenarios. 

•  Peacetime  operational  constraints  could  permit  optimization  by  AMC  as  well. 


Pilots 

The  objective  of  our  first  meeting  with  AMC  pilots  was  to  conduct  a  task  analysis  of  pilot 
activities  before  and  during  a  mission  to  learn  how  they  accomplish  a  mission  and  how  they 
address  fatigue.  Also  to  leam  how  a  fatigue  analysis  tool  might  help  in  their  rescheduling  and 
how  it  could  fit  seamlessly  into  their  re-planning  processes. 

The  meeting  was  held  on  October  13,  2006  from  0800-1230.  The  Chief  of  Safety  for  the  68th 
Airlift  Squadron  (AS),  introduced  the  F-PAS  development  team  to  the  four  participating 
pilot/SMEs.  In  his  introductory  remarks,  he  indicated  that  the  current  AMC  commander  had 
stated  that  AMC  could  not  continue  as  it  has  for  decades  with  crews  flying  fatigued.  “He  wants 
change.” 

After  SMEs  introduced  themselves,  the  goals  and  objectives  of  the  meeting  were  stated  with  an 
opportunity  for  questions.  We  discussed  fatigue  management  and  asked  the  pilots  about  their 
experience  with  fatigue.  We  discussed  the  SAFTE™  model  and  answered  questions.  We  then 
presented  a  FAST™  analysis  of  an  AMC  mission  from  a  PowerPoint  slide  put  together  by  a  pilot 
from  the  68th  AS,  to  get  the  SMEs  to  thinking  about  fatigue.  Generally,  the  pilots  liked  the  ideas 
presented  for  evaluating  schedules  for  their  impact  on  fatigue,  but  felt  that  it  would  not  make  a 
difference  in  how  missions  were  scheduled  unless  the  Tanker  and  Airlift  Command  Center 
(TACC)  schedulers  at  Scott  AFB  were  using  the  tool  also. 

Thus,  the  meeting  did  not  take  the  nonnal  course  and  move  into  a  task  analysis  as  explained  in 
the  results  section.  The  meeting  was  tenninated  shortly  after  1200  without  a  task  analysis,  but 
with  a  better  understanding  of  how  AMC  missions  are  scheduled  and  executed.  The  scheduling 
process  as  described  by  the  pilots  is  described  in  the  results  section  followed  by  notes  from  the 
development  team. 

Flight  Surgeons 

The  first  requirements  assessment  session  for  AF  flight  surgeons  (FS)  was  conducted  under  the 
combined  auspices  of  the  USAF  School  of  Aerospace  Medicine  (USAFSAM),  the  Residency  in 
Aerospace  Medicine  (RAM)  Program  of  USAFSAM,  and  the  Biobehavioral  Performance 
Branch  of  the  Air  Force  Research  Laboratory  (AFRL  711  HPW/RHPF),  Brooks  City-Base, 
Texas.  The  session  was  held  at  USAFSAM  in  the  RAM  conference  room  on  7  September  2007. 
The  director  of  the  RAM  program  (USAFSAM/GE)  and  14  RAM  students  participated  in  the 
discussions  as  subject-matter-experts  (SME).  Nearly  all  of  the  SMEs  were  aware  of  FAST™, 
but  only  three  had  actually  used  it  to  assess  fatigue  in  an  operational  setting. 

After  the  investigators  introduced  themselves,  they  stated  their  goals,  objectives,  and  agenda  for 
the  meeting  and  answered  questions.  With  only  90  minutes  available  to  meet  with  the  RAMs  for 
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a  requirements  assessment,  an  abbreviated  agenda  was  used  in  the  hope  of  acquiring  information 
necessary  to  begin  the  design  of  the  interface.  Our  plan  was  to  familiarize  the  FS  with  FAST™, 
have  them  provide  an  overview  of  their  tasks  and  background,  introduce  the  task  analysis  (TA) 
procedures,  and  then  conduct  the  task  analysis.  As  we  worked  our  way  through  FAST™ 
capabilities,  it  became  clear  that  TA  was  not  really  needed  since  the  FS  did  not  want  an  interface 
specifically  designed  for  them  to  explore  fatigue  countenneasure  options.  This  is  explained  in 
the  results  section  that  highlights  the  ideas  and  conclusions  of  the  session.  Since  the  FS  did  not 
need  an  introduction  to  fatigue  and  performance,  we  focused  on  how  using  a  model  such  as 
FAST™  could  provide  quantitative  data  and  projections  in  time  about  how  sleep  and  circadian 
rhythm  predicted  performance  capability.  Concepts  from  the  SAFTE™  model  and  FAST™ 
graphs  (Eddy  &  Hursh,  2001,  2006b)  were  used  to  illustrate  fatigue  and  perfonnance  effects. 

WALK-THROUGH 

From  the  user  task  descriptions  and  the  requirement  assessment  (RA),  we  created  scenarios  for 
user  testing  and  review.  In  the  walks  through  the  draft  interface,  the  investigators,  potential  end 
users,  and  evaluators  worked  together  to  step  through  typical  tasks  for  which  the  interface  was 
designed.  Because  the  SME  users  were  flight  surgeons  and  were  familiar  with  fatigue 
management  terminology  and  fatigue  effects,  they  were  only  given  preliminary  training  on  the 
capabilities  of  the  new  browser-based  tool  and  on  its  draft  mission-planning  interface.  The 
investigators  demonstrated  the  tool  by  entering  data  for  a  fictitious  scenario.  Questions  and 
discussions  accompanied  each  screen  of  the  tool. 

We  estimated  roughly  the  total  number  of  usability  problems  in  the  interface  from  the  number  of 
problems  (E)  identified  during  the  walk-through.  Assuming  a  detection  rate  of  about  30%  in  the 
walk-through,  the  total  number  of  problems  would  be  about  equal  to  E  divided  by  0.30  (Bailey, 
1997). 

INSPECTION  EVALUATION 

One  inspection  evaluation  was  conducted  by  NTI  under  the  auspices  of  the  Altitude  Training 
group  of  USAFSAM  (USAFSAM/ATA)  and  was  held  in  their  computer  laboratory  at  Brooks 
City-Base,  San  Antonio,  Texas.  In  the  inspection  evaluation,  three  flight  surgeons  with  7,  8,  and 
14  years  of  active  duty  from  the  USAFSAM  Residency  in  Aerospace  Medicine  (RAM)  program 
served  as  SMEs.  They  tried  to  do  typical  tasks  with  the  interface  while  a  investigator  watched, 
listened,  and  took  notes.  We  wished  to  identify  usability  problems,  collect  quantitative  data  on 
SMEs’  perfonnance,  and  detennine  the  SMEs'  satisfaction  with  the  product.  More  specifically, 
we  wished  to  learn: 

•  Could  the  SMEs  complete  the  relevant  tasks  successfully? 

•  How  long  did  it  take  the  SMEs  to  do  each  subtask? 

•  Did  the  SMEs  perfonn  well  enough  to  meet  their  usability  objectives? 

•  How  satisfied  were  the  SMEs  with  the  interface? 

•  What  changes  were  needed  to  make  sure  that  the  interface  would  enable  more  users  to 
perfonn  more  successfully? 
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These  latter  questions  were  addressed  through  the  questionnaire  shown  in  Appendix  A.  The 
observer  collected  detailed  data  using  the  scoring  sheet  shown  in  Appendix  B.  The  user  was 
provided  with  a  separate  document  showing  the  34  questions  in  the  scoring  sheet. 

One  goal  of  the  contract  effort  was  to  allow  schedulers  to  apply  fatigue  management  to  AMC 
missions  through  the  incorporation  of  our  tool  into  the  workflow  of  their  scheduling  tasks.  The 
tool  we  are  creating  for  them  can  evaluate  the  impact  of  a  mission  schedule  on  the  crew’s  ability 
to  perform  it,  considering  their  opportunities  for  sleep  before  and  during  the  mission.  Our  goal 
was  that  90%  of  AMC  schedulers  using  our  computerized  fatigue  management  tool  should  be 
able  to  do  90%  of  their  fatigue-analysis  tasks  reasonably  well  with  relatively  little  training 
beyond  an  introductory  overview  of  our  product. 


12 

Approved  for  public  release;  distribution  unlimited,  Public  Affairs  Case  File  No.  09-402, 
August  21,  2009,  Brooks  City-Base,  Texas. 


RESULTS 


REQUIREMENTS  ANALYSIS 
AMC  Schedulers 

To  understand  the  scheduler  who  might  use  our  tool,  we  first  evaluated  the  answers  to  our  initial 
questions  about  the  potential  users. 

Who  would  the  product  users  be,  who  should  be  included  and  who  should  be  excluded? 

For  the  AFRC  SMEs,  the  schedulers  were  generally  pilots  who  were  working  in  the  Wing  HQ 
when  a  mission  request  arrived.  Pilots  are  familiar  with  everything  necessary  to  plan  a  mission 
and  they  are  most  aware  of  the  issues  the  aircrew  would  confront  while  in  the  field.  AMC  users 
would  be  senior  enlisted  personnel  or  lieutenants  at  TACC.  Their  experience  would  be  less  than 
a  pilot  at  a  Reserve  wing,  but  they  would  have  greater  resources  to  draw  upon  in  planning  the 
mission.  The  TACC  has  a  comprehensive  web-based  program,  the  Global  Decision  Support 
System  (GDSS),  that  links  into  many  databases  and  information  sources  that  facilitate  planning  a 
mission.  Enlisted  personnel  E-5  and  below  will  not  likely  plan  missions  or  use  the  fatigue 
management  product. 

Generally,  what  knowledge,  skills  and  abilities  did  schedulers  need  to  accomplish  mission 
planning? 

Most  schedulers  are  college  graduates  and/or  able  to  read  and  write  at  the  college  level.  They 
need  those  skills  because  of  the  considerable  requirements  for  finding  information,  reading  and 
understanding  it,  and  applying  it  to  the  planning  task. 

What  special  skills/abilities  do  schedulers  need  for  Scheduling/Planning  missions? 

Schedulers  must  understand  the  requirements  and  limitations  of  the  aircraft  for  which  they  are 
planning.  Such  as  items  as  fuel  capacities  and  burn  rates,  distance  limitations,  runway  lengths, 
hauling  capacities,  show  times,  loading  and  unloading  times,  post  flight  procedure  times,  etc. 
They  must  know  the  aircrew  skill  and  training  requirements  for  the  various  aircraft  and  for  the 
various  cargoes.  They  must  understand  air  refueling  rules  and  how  to  get  tankers  into  position 
for  air  refueling.  Schedulers  must  know  about  base  and  airport  opening  and  closing  times, 
restrictions,  and  each  country’s  requirements  for  accommodating  US  military  aircraft. 

What  training  do  schedulers  need  to  do  their  mission-planning  job? 

Schedulers  acquire  the  skills  and  knowledge  listed  above  and  other  knowledge  through  AMC 
training  courses  and  on-the-job  training.  Pilots  get  this  infonnation  in  the  course  of  their  initial 
and  continuing  aviation  training. 

The  participant  SMEs  were  asked  to  help  us  construct  a  tabular  display  of  the  realistic  planning 
goals,  processes,  and  tasks  that  they  actually  accomplished  in  designing  a  mission  schedule.  In 
the  TA,  the  investigators/interviewers  recorded  the  goals,  processes,  and  tasks  described  by 
schedulers  as  they  constructed  a  mission  schedule.  The  information  was  displayed  in  a  table 
projected  on  a  large  screen  in  real  time  that  all  could  see.  The  first  column  of  the  table  was  for 
goals  and  processes,  the  second  for  tasks.  This  allowed  the  investigators  to  correct  the 
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information  in  real  time.  At  the  end  of  the  day,  SMEs  reviewed,  amended  and  corrected  the 
recorded  data. 

While  our  goal  was  to  understand  how  the  schedulers  create  their  mission  schedules,  a  secondary 
goal  was  to  see  if  FAST™  features  matched  up  with  their  tasks  in  such  a  way  that  they  could  use 
FAST™-like  output  to  prevent  or  reduce  the  fatigue  on  aircrew  while  still  planning  a  successful 
mission.  Table  1  contains  the  TA  and  FAST™  usability  chart  produced  with  the  scheduler’s 
inputs.  It  shows  the  resulting  Goals,  Processes,  and  Tasks  created  during  the  TA.  As  one  moves 
down  the  rows  of  the  table  in  the  mission  planning,  time  increases.  Data  in  the  columns,  from 
left  to  right,  include  global  processes  and  goals  and  then  the  specific  tasks  that  were  directly 
related  to  those  stated  processes  and  goals.  From  this  analysis,  we  can  see  that  the  schedulers 
articulated  eight,  mostly  independent,  top-level  processes  or  goals.  These  eight,  top-level 
processes  were: 

1 .  User  or  Mission  Requirements  (what,  where,  when) 

2.  Resource  Availability  (aircraft,  aircrew,  destination  suitability) 

3.  Constraints/Regulations/Waivers  (schedule,  Ops  restrictions,  airspace,  servicing, 
clearances) 

4.  Operational  Risk  Management  (covers  many  tasks) 

5.  Command  &  Control  IPS  Input  (mission  details) 

6.  Task  Support  Agencies  (purser,  XP/Mx,  APS,  life  support,  Ravens) 

Definitions: 

■  Purser 

■  XP/Mx 

■  APS  -  Advanced  Planning  and  Scheduling  (APS)  Pathfinder  is  an  off-the-shelf 
technology  used  for  supply  chain  planning  and  decision  support  functions. 

■  Life  Support 

■  Ravens  -  Are  a  mobile  force  protection  agency  that  provides  aircraft  security  in 
unsecured  ground  locations  while  a  mission  is  enroute. 

7.  Aircrew  Requirements  (augmented,  special  qualifications,  mission  commander,  extra 
crew) 

8.  Supervisory  Approval  (for  training  missions) 
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Table  1.  The  Goals,  Processes,  and  Tasks  of  AMC  Schedulers 


PROCESSES/GOALS  STEPS 

TASKS  TO  ACCOMPLISH 

SUGGESTED  FAST™ 
MODIFICATIONS 

1 .  User  (Mission?) 

Read  &  understand  tasking 

Direct  capture  from  GDSS  for 

requirements 

(Annex  C) 

missions  TACC  has  already  planned 

what 

where 

Revalidate/evaluate  timely 

Major  Weapon  System  (MWS) 

when 

tasking  (72hr  notice?  24hrs?) 
Capacity,  capability, 

default  parameters 

resources,  dynamics, 

Master  program  for  #  of  legs,  alert 

via; 

complexity, 

defaults,  and  rest  period  minimums, 

JAATT  planning  conference 

compatibility,. . .  .etc. 

Augmented  (naps)  vs.  Basic  (no 

ARMS  (air  refueling  rnsns) 

naps)  crew  rules,  and  critical  events 

AFRC  AF  Reserve 

Airfield(s) 

with  drag  &  drop  capability.  Is  the 

Command)  allocations 

above  backwards  on  naps? 

OST  (order  &  ship  time) 

Latest  Allowable 

requirements 

Pickup/Delivery  (LAP/LAD) 

Allow  user  defined  mission 
constraints  (ex:  NOTAM  or  quiet 
hour  restrictions,  LAP,  LAD, 
waivers,  etc.)  that  indicate  conflicts 
but  can  be  overridden 

Does  any  of  the  above  duplicate 
other  existing  software  program 
capabilities? 

2.  Resource  availability 

special  configurations  or 

(major  elements) 

modifications  directed/req’d 

aircraft 

aircrew 

tail  tasking  (level/availability) 

destination  suitability 

special  quals  (crew,  MEGPs, 
Mx,  Ravens,  Purser,  etc.) 

AATS 

API/Giant  report  suitability 
Ops  Hours/ restrictions 
NOTAMS/PPR 

Weight  Bearing  Capacity 
Ground  Support 

MHE  &  personnel 

Expected  sleep  facility  quality 

Rest  facilities  avail/suitability 

ratings;  excellent,  good,  fair,  poor 
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Table  l(continued).  The  Goals,  Processes,  and  Tasks  of  AMC  Schedulers 


PROCESSES/GOALS 

STEPS 

TASKS  TO  ACCOMPLISH 

SUGGESTED  FAST™ 

MODIFICATIONS 

3.  Constraints 

/regulations/waivers 

(details) 

Schedule 

Ops  restrictions 
quiet  hours 
day/night 

Airspace  (ALTRV) 
Servicing 

Dip  Clearances 
/FCG 

Apply  rules  and  regulations  to 

optimally  plan  mission 

Crew  rest/duty  day/tactical 

events/transition  training 

Consider  stage/crew  options 

ops  restrictions 
ground  times  (Velocity  Init?) 
quiet  hours 
day/night 

Airspace  (ALTRV) 

Servicing 

Dip  Clearances/FCG 

CFP  from  TACC 

Flight  planning  calculations 
Winded  leg  lengths,  AR 
duration,  etc. 

Duty  day 

BRAVO  standby 

User  defined  mission  constraints  monitor 
user  inputs  (ex:  NOTAM  or  quiet  hour 
restrictions,  LAP,  LAD,  waivers,  etc.) 
and  flag  conflicts  but  can  be  overridden 
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4.  ORM 

worksheet/ matrix/ checklist 

Color  coded  mean  effectiveness  for 

long  vs  short  days 

overall  mission  and  for  critical  events 

augmented  vs  basic 

Clean,  user-defined  summary  report 
output  to  aircrew  scheduler/aircraft 

departure  &  arrival  time  of  day 

mission  complexity/difficulty 

commander 

combat? 

Note; 

Tactics? 

current  output  in  minutes  should  covert  to 

AR? 

hours  +  minutes  (15  minute  increments) 

Formation? 

Drag  &  drop  all  objects  (sleep,  work,  and 

Cargo  hazards? 

events) 

Crew  rest 

Right  click  to  modify  object  attributes  or 

Duration 

times 

Facilities  suitability 

Mouse  over  to  read  object  detail 

Meals 

Transportation 

Future  enhancements: 

Crewmember  1,  2  etc.  assessments 

Click  and  drag  curve  to  improve 
performance  and  suggest  fatigue  risk 
mitigation  (ie:  meds,  crew  rest,  task 
reduction,  etc) 

Display  choices  for  illustration  capture 
(ex:  all  data  (cluttered),  1  level  de-clutter 
(critical  events  only),  and  another  level 
declutter  (basic) 
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Table  l(continued).  The  Goals,  Processes,  and  Tasks  of  AMC  Schedulers 


PROCESSES/GOALS  STEPS 

TASKS  TO  ACCOMPLISH 

SUGGESTED  FAST™ 
MODIFICATIONS 

5.  C2  IPS  input 

Enter  mission  specifics 
(type  into  computer 
program) 

Itinerary 

Remarks 

Push  to  GDSS 

FAST™  score  summaries 

Whole  mission 

Critical  events 

Low  performance  times 

6.  task  support  agencies 

Purser 

XP/Mx 

APS 

Life  support 

Ravens 

6.  Communicate 
e-mail 
phone 

Air  Order  of  the  Day  (AOD) 
Determine/specify 
priorities 

meeting 

required  changes? 
final  paper  copy 

Command  Post  (execution 
phase)  access  to  file  for 
modification  and  new  FAST™ 

assessment 

7.  Aircrew  requirements 
Augmented? 

Special  qual? 

Mission  Commander? 

Extra  crew/DH? 

Determine  taskable  squadron(s) 
Special  Qual 

Bean  count 

Bookie  1 .0  (access  database) 
Pain  level 

A  ATS  website  (TACC) 

8.  Supervisory  Approval  for 
training  missions 

AFRC  (if  applicable) 

Unit  DO 

User  defined  reports 

Suggest  possible  mitigation 
options? 

Notes:  This  Table  starts  with  mission  planning  and  progresses  through  to  the  end  of  mission  planning.  Data  in  each 
column  move  from  left  to  right,  from  global  Process/Goals,  to  specific  Tasks  that  are  directly  related  to  those  stated 
processes/goals.  The  last  column  addresses  how  the  current  FAST™  related  to  the  tasks  that  met  the  goals  of  the  mission. 
The  phrases  in  italics  were  considered  particularly  important. 

The  tasks  for  these  processes  and  goals  vary  widely.  Some  include  verifying  the  information 
received  from  TACC  with  the  mission  order,  looking  up  the  latest  information  on  airfields  and 
aircraft,  considering  optional  solutions  to  accomplish  the  mission,  assessing  the  complexity  of  a 
mission,  crew  rest  opportunities,  and  a  commander’s  idiosyncrasies.  Column  2  in  Table  1  lists 
all  the  tasks  associated  with  the  top-level  processes  and  goals. 

Several  of  FAST™’s  capabilities  fulfill  the  needs  of  the  schedulers,  but  other  requirements  are 
lacking.  For  example,  FAST™  nicely  forecasts  perfonnance  effectiveness  for  an  entire  mission, 
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critical  events,  and  highlights  low  perfonnance  times.  However,  it  would  be  a  big  improvement 
if  FAST™  could  directly  capture  a  mission  from  GDSS  when  TACC  has  already  planned  a 
mission  instead  of  requiring  a  scheduler  to  key  all  the  infonnation  into  the  fatigue  analysis  tool. 


User  Design  Recommendations  and  FAST™  Usability 

The  recommendations  coming  from  the  TA  as  summarized  by  the  investigators  follow. 

1 .  Quick  Manual  Entry:  As  of  this  writing,  a  mission  cannot  be  read  directly  into  FAST™ 
(NTI  has  not  yet  been  given  the  file  export  capabilities  of  GDSS)  for  our  new  web-based 
tool.  Therefore,  the  interface  should  be  designed  to  allow  a  scheduler  to  quickly  and 
easily  build  a  mission  directly  using  pre-existing  mission  legs  (T/O  (takeoff),  Critical 
Events  (CE;  purpose  of  stop),  Aerial  Refueling  (AR),  and  Touchdown  (TD)  -  see  more 
about  this  in  the  next  section).  The  scheduler  filling  in  the  infonnation  required  in  a 
general  template  that  could  be  expanded  or  shrunk  could  assemble  the  mission.  Using  a 
general  template  that  could  be  expanded  or  shrunk  to  fit  any  mission  requirements,  a 
scheduler  could  assemble  a  mission  by  filling  in  the  required  information. 

•  A  longer-term  goal  should  be  to  import  the  mission  tasking  directly  from  GDSS  data, 
with  little  additional  editing.  Direct  capture  from  GDSS/GDSS-2  for  AMC-directed 
missions  (TACC)  should  be  planned  for  the  future.  Another  alternative  would  be  to 
export  a  data  file  from  GDSS  and  import  it  for  an  automatic  schedule  build  in  the 
fatigue  analysis  tool. 

2.  In  this  conceptualization,  a  master  screen  showing  a  mission  template  would  be  available 
on  the  display  that  has  placeholders  for  mission  events  and  entry  boxes  for  T/O,  CE,  AR, 
and  TD  times. 

•  If  an  enroute  stop  occurs,  then  it  should  be  possible  to  input  and  recall  the  purpose  of 
the  stop  easily.  Generally,  an  entry  fonnat  should  match  mission  fragments  for  ease 
of  use. 

The  program  would  ask,  “How  many  proposed  legs  should  there  be?”  A  leg  is  a  general 
term  that  a  user  defines  by  selecting  a  platfonn  (aircraft  type).  Schedulers  for  different 
platforms  may  use  different  tenninology  for  the  equivalent  of  a  leg.  In  the  software,  a  leg 
should  be  an  object  that  can  contain  the  following  other  objects  all  with  unique  icon 
types:  T/O,  AR,  CE,  and  TD.  Legs  may  have  only  one  T/O  and  one  landing,  but  may 
have  any  number  of  ARs  or  CEs  including  none.  A  leg  can  have  a  show  time  and  post 
landing  time  as  well.  T/Os,  landings,  and  ARs  have  a  latitude  and  longitude  (lat/lon), 
date,  and  time.  CEs  have  a  date  and  time. 

Figure  2  is  a  schematic  example  of  the  objects  in  a  mission.  Mission  XYZ  is  composed 
of  two  legs  each  with  various  elements,  but  always  having  one  T/O  and  one  TD.  Notice 
the  legs  each  have  a  beginning  and  an  end.  They  are  independent  objects  that  are 
contained  within  the  mission  object,  “XYZ.” 
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MISSION  “XYZ” 

LEG  1 

T/O  (lat/lon,  date/time) 

AR1  (lat/lon,  date/time) 

AR2  (lat/lon,  date/time) 

CE1  (date/time) 

AR3  (lat/lon,  date/time) 

TD  (lat/lon,  date/time). 

END  LEG1 
LEG  2 

T/O  (lat/lon,  date/time) 

AR1  (lat/lon,  date/time) 

CE1  (date/time) 

AR3  (lat/lon,  date/time) 

TD  (lat/lon,  date/time). 

END  LEG2 

END  MISSION  “XYZ” 

Figure  2.  Schematic  of  AMC  Mission  Legs  (Segments). 

The  following  were  discussion  points  for  developing  requirements. 

•  We  are  using  the  term  leg  here  generally,  since  bomber  crews  flying  long-range 
missions,  or  fighters  doing  multiple  mission  elements  may  want  the  term  “leg” 
defined  generally,  and  not  specifically  as  a  T/O  or  TD  flight  segment.  Future 
discussion  point:  The  group  talked  about  specifying  the  air  platfonn  on  the  initial 
data  entry  screen.  If  we  could  use  that  infonnation  to  define  the  terminology,  we 
could  implement  this  suggestion.  That  is,  the  platform  or  airframe  would  define  the 
meaning  of  “leg.” 

•  Allow  for  crew  changeover  and  re-computation  when  augmented  crews  are  used,  etc. 
Future  discussion  point:  It  might  be  better  to  have  an  easy  way  to  port  the  mission 
components  from  one  schedule  to  another  and  then  add  the  sleep  histories  of  the 
augmented  crew  to  the  new  schedule  and  continue  from  there  when  augmented  crews 
are  used. 

•  Need  to  make  provision  for  “Crew  Standby  Duty” 

•  The  implied  “protected  times”  before  the  T/O  and  after  TD  should  be  user-defined  as 
a  part  of  the  initial  entry  since  time  is  needed  for  mission  planning,  review,  preflight, 
etc,  and  a  multitude  of  post-landing  tasks.  A  scheduler  should  be  able  to  input  and 
save  standard  pre-and  post-flight  periods,  but  be  able  to  modify  them  easily  if  needed, 
perhaps  with  a  mouse-over  and  then  “right  click”  for  a  sub-menu.  For  example,  after 
entering  the  platform/mission-type,  the  software  might  present  the  scheduler  with  the 
defaults  for  the  platform/mission-type  selected.  These  could  be  modified  then  or  later 
as  desired. 

3.  The  scheduler  needs  the  capability  to  label/edit  the  key  events  in  a  flight  leg,  such  as  T/O, 

AR’s,  CE,  and  TD  at  first-  and  second-order  levels.  First,  during  the  master  schedule 

development  as  “Take  off”  with  airport  Symbol,  etc.;  then,  the  user  may  add  details  to  a 
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sub-screen  associated  with  that  object  in  terms  of  remarks  with  mouse-over.  For 
example,  select  the  Takeoff  Icon  on  the  screen,  and  then  perhaps  a  right  click  to  add  or 
see  additional  details  (“T/O  from  XXX  at  0400  at  a  blacked-out  field  with  SAM  threats  in 
the  area). 

•  After  all  of  the  legs  and  critical  events  in  the  master  schedule  are  entered,  the  user 
should  be  able  to  look  at  a  first-cut  fatigue  analysis  to  see  what  the  average 
effectiveness  is  per  defined  leg  or  a  whole  mission.  In  addition,  an  analysis  for  key 
subsets  of  a  mission  by  highlighting  the  area  on  the  graphic  user  interface  (GUI),  and 
clicking  to  get  the  effectiveness  between,  for  example,  “waypoints.”  Also,  the  lowest 
area  of  effectiveness  in  the  leg  and  mission  that  is  computed  should  be  called  to  the 
user’s  attention  if  it  is  below  a  certain  level.  This  value  may  be  defined  by  the  user  at 
the  master  level,  or  selected  and  changed  to  evaluate  different  scenarios. 

4.  With  the  basic  mission  attributes  in  place,  the  performance  charts  generated,  the  key 
waypoints  and  CEs  labeled  appropriately — there  is  a  need  to  evaluate  “what  if’  scenarios. 
This  should  be  as  easy  for  the  scheduler  as  possible. 

5.  The  key  attribute  of  the  GUI  should  be  “drag-and-drop”  editing  of  all  objects. 

•  The  objects  defined  above  (Legs  T/O,  AR,  CE,  TD)  should  be  movable  in  time  by  an 
object-oriented  drag-and-drop  editor.  The  tool  should  then  re-compute  performance 
effectiveness  and  display  new  charts.  (Furthennore,  when  an  object  is  grabbed, 
dragged  and  dropped,  it  should  be  possible  to  access  the  original  table  values  with 
one  or  two  clicks.  Thereafter,  the  user  should  be  asked  if  they  want  the  changes  made 
pennanent  based  upon  the  new  scenario.  An  alternative  would  be  to  save  the  edited 
schedule  as  Scenario  2.  In  any  case,  it  should  be  possible  to  save  the  edited  schedule 
as  Scenario  2. 

o  There  are  some  constraints  to  be  considered  here.  1 .  The  model  takes  several 
seconds  to  compute  the  new  values  even  for  a  short  schedule.  Therefore,  we 
may  want  to  be  able  to  make  several  changes  and  then  click  a  re-compute 
button  (you  can  do  this  in  Excel  for  large  spreadsheets).  2.  Saving  a  modified 
schedule  is  always  an  option,  but  we  don’t  want  it  to  be  done  automatically. 
Huge  amounts  of  data  would  be  saved  and  the  user  may  not  want  95%  of  it. 
With  current  FAST™,  a  user  can  save  the  schedule  at  any  time,  can  rename  a 
modified  schedule,  and  can  save  it,  etc.  That  forces  the  user  to  decide  what  he 
wants  to  save  each  time  it  is  done  by  assigning  it  a  new  file  name.  The  undo 
command  should  be  capable  of  storing  up  to  three  changes.  That  would  give  a 
user  enough  memory  depth  for  correcting  mistakes  or  trying  something  and 
then  returning  to  the  original  when  it  didn’t  work. 

•  The  tools  should  be  able  to  evaluate  at  least  five  different  mission  scenarios  based  on 
the  original  baseline  mission  values.  The  alternate  scenarios  need  basic  name  change 
labels/reminder  boxes  one  can  edit.  This  is  so  that  they  can  be  reminders  of  the 
potential  changes  to  the  baseline  mission  structure. 

o  File  names  and  descriptions  accomplish  this  purpose  in  FAST™. 

•  The  fields  on  the  GUI  that  contain  sleep  and  work  should  be  editable  (shortened  or 
extended)  without  leaving  the  main  display.  Performance  effectiveness  values  should 
be  computed  after  a  move  is  made.  (Note:  It  should  be  remembered  that  only 
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changes  to  sleep  or  waypoints  require  a  re-compute  using  the  SAFTE  model. 
Changing  work  intervals  or  critical  points  have  no  effect  on  the  model  of  the  human 
simulation.) 

•  The  details  of  an  object  should  display  when  the  mouse  pointer  rests  over  them. 

•  Generally,  a  right  click  should  allow  modification  of  an  object’s  attributes  or  times 
(with  modification  possible). 

•  If  the  schedule  is  for  multi-crewed  aircraft,  then  need  the  capability  to  assess 
Crewmember  1,2,  Loadmaster,  etc. 

•  The  time  should  be  displayed  in  the  fonnat  of  hours  +  minutes.  Also,  allow  for  15 
min  increments  for  the  drag  and  drop  functions  on  the  main  GUI  display  with  the 
ability  for  one -minute  resolution  for  direct  or  right  click  pop-up  inputs. 

•  Use  color/shape  coding  for  different  types  of  events:  T/O,  AR,  CE,  TD,  and  pre-  and 
post- flight  activities. 

•  Eventually:  Click  and  drag  curve  to  improve  perfonnance  and  suggest  fatigue  risk 
mitigation  (crew  rest/naps,  task  reduction,  and  medical  countenneasures).  Request 
decision-aid  maximizes  performance  for  a  particular  time  point.  This  would  have  to 
be  limited  to  making  changes  constrained  to  within  a  variable  number  of  hours  of  the 
particular  time. 

•  Display  mission  goal  criterion  line  and  actual  perfonnance  effectiveness  line  for  the 
work  schedule  (include  color  coding) 

•  Remind  personnel  that  if  mean  effectiveness  values  are  poor,  then  additional  crew 
rest  and/or  changing  mission  parameters  need  consideration  as  part  of  an  Operational 
Risk  Management  (ORM)  approach.  (Narrative  reports  regarding  parts  of  a  schedule 
falling  below  criteria  could  be  used  for  this  purpose.) 

6.  Perfonnance  effectiveness  displays  and  reminders  are  needed  for  crews  while  they  are 

flying.  Perfonnance  effectiveness  score  summaries  need  to  be  presented  for  the  whole 

mission,  and/or  just  the  mission  “legs”  as  desired. 

•  At  the  very  beginning,  the  ability  to  go  backwards  in  time  “X  days”  will  be  needed  to 
better  assess  the  fatigued  state  of  the  pilot  before  the  mission  begins  as  an  additional 
option. 

•  Calculate  and  display  color-coded  mean  effectiveness  curves/values  for  the  overall 
mission  and  critical  events 

•  Display  goal  criterion  line  and  actual  criterion  line  for  mean  work  schedule 
effectiveness 

•  Eventually  be  able  to  modify  the  mission  values  during  the  mission  on  board  an 
aircraft  easily  to  change  parameters  to  reflect  the  real  world  events  to  see  how  the 
effectiveness  values  change  (by  taking  an  extra  nap,  or  encountering  worse  than 
expected  headwinds,  for  example). 

•  Clean,  user-defined  summary  report  output  to  aircrew/scheduler/  aircraft  pilots 

o  Include  Graph/Tables/Text  all  on  one  page 

o  User  defined  reports  using  a  checklist  similar  to  the  existing  FAST™  timeline 
report. 

o  Command  Post  (execution  phase  access  to  file  for  modification  and  new 
FAST™  assessment  if  needed) 

o  Some  files—  Read  only 
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7.  Additional  details 

•  User  should  have  the  option  of  entering  an  airport  symbol,  or  coordinates,  as  the  need 
dictates.  Or,  making  changes  to  an  airport/location  “on  the  fly”  simply  by 
highlighting  the  location  and  right  clicking  to  pull  up  a  box  with  coordinates/symbols 
in  it. 

•  The  Autosleep/ Autonap  features  need  to  work  for  transmeridian  flights. 

•  The  ability  to  insert  easily  a  nap  during  a  work  schedule  is  important  to  see  its  effect 
on  performance.  A  minimum  nap  of  15  min.  is  probably  ok,  but  longer  ones  need  to 
be  more  specific  than  15,  30,  or  45  minutes. 

•  Need  ICAO  identifiers  in  database  for  more  ease  of  calling  up  locations. 

•  Need  to  consider  “BRAVO  standby.”  This  disastrous  mechanism  leads  directly  to 
pilots  flying  fatigued.  We  need  to  give  this  some  thought. 

•  Need  display  choices  for  infonnation  capture:  All  data,  1  level  de-clutter  (key  events 
only),  and  another  level  de-clutter  for  basic  infonnation. 

The  major  recommendations  coming  from  the  usability  analysis  follow.  These  are  general 
comments  on  the  FAST™  interface,  Version  1.0.3,  by  the  schedulers  after  learning  it  and  using  it 
for  about  one  hour. 

1 .  Fine  technical  tool,  but  too  hard  to  use  in  practice  without  specialized  training. 

2.  Graphical  User  Interface  needs  major  overhaul  for  improved  ease-of-use  for  schedulers, 
planners,  and  other  risk-assessment  personnel  as  aircraft  missions  are  planned. 

3.  Current  interface  in  general  takes  too  much  time  for  mission  input,  and  it  is  difficult  to 
use  if  changes  or  “what-if  ’  scenarios  need  to  be  examined. 

4.  Ideally,  need  to  be  able  to  take  data  that  comes  to  schedulers  and  planners,  input  it  with 
little  modification  into  FAST™,  and  get  out  preliminary  result  of  fatigue -related 
computations. 

5.  An  “object-oriented”  approach  to  the  interface  needs  to  be  developed  directly  off  of  the 
main  FAST™  GUI  screen,  with  the  ability  to  get  into  the  details  of  the  program  if  needed 
by  calling  up  sub-screens. 

6.  Much  easier  evaluation  and  changing  of  Mission  Legs  and  “what  if’  scenarios  directly 
from  the  FAST™  Output  GUI  should  be  developed. 

A  caveat  for  the  AMC  TA  results  and  suggestions  for  improving  FAST™:  there  are  differences 
in  AMC  between  the  regular  AF  and  AFRC.  Although  these  differences  are  not  minor,  we 
believe  that  the  guidance  and  recommendations  we  received  from  our  Reservist  schedulers  was 
exactly  what  was  needed  to  make  the  F-PAS  product  more  useful  to  both.  If  anything,  Reservists 
have  more  freedom  to  plan  missions  and  make  changes  that  may  impact  fatigue  and,  therefore, 
perfonnance.  With  the  additional  freedom,  they  require  more  decision-aid  capabilities  to  edit 
schedules,  play  “what-if’  scenarios,  and  creatively  attempt  to  maximize  perfonnance  and 
minimize  fatigue  in  their  AFRC  missions.  The  TA  development  team  recognizes  that  AFRC  and 
AMC  might  have  different  needs,  but  believe  that  we  should  use  AMC  to  establish  common 
ground  regarding  FAST™  implementation,  and  then  identify  exceptions  for  AFRC  mission 
planners. 
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From  other  sources.  As  FAST™  was  developed,  it  was  made  available  to  those  who  wanted  it, 
in  return  for  their  feedback  and  suggestions  for  improving  it.  In  addition  to  the  TA  and  usability 
analysis  conducted  with  the  AMC  schedulers,  commercial  users  and  other  AF  users  have 
provided  suggestions  to  improve  the  tool.  Suggestions  that  duplicate  AMC  recommendations  or 
that  have  already  been  addressed  in  the  current  version  of  FAST™  are  not  reported  here. 

A  Major  from  3  AS/DOIP  (AMC),  Dover  AFB,  had  used  an  earlier  version  of  FAST™  for  about 
six  months  and  gave  us  the  following  excellent  feedback  in  September,  2004.  A  USAF  Captain 
gave  similar  recommendations.  Many  of  their  suggestions  parallel  those  of  the  AMC  schedulers 
coming  out  of  the  TA.  The  following  is  taken  from  a  series  of  emails  between  the  users  and  Dr. 
Eddy.  Occasionally  items  are  clarified  or  potential  solutions  are  proposed  for  further 
elaboration. 

1 .  I  would  like  to  be  able  to  select  multiple  events  and  move  or  shift  them  one  way  or 
another  and  see  the  effect  as  I  move  them  (such  as  moving  a  3 -hour  nap  forward  or 
backward,  or  delaying  a  mission  for  several  hours). 

I  would  suggest  that  this  change  be  made,  since  the  tool  is  otherwise  quite  useless  to 
AMC,  since  our  missions  slip  in  time  so  often,  and  we  have  to  play  what  if  s  with  our 
naps.  Bottom  Line,  I  need  to  be  able  to  generate  an  entire  profile  including  all  data  entry 
in  30  Seconds.  Otherwise,  our  schedulers  will  not  use  it. 

2.  Does  the  model  predict  sleep  pressure  (the  circadian  force  compelling  one  to  sleep)? 

It  needs  to  be  plotted,  since  this  tells  us  when  we  need  to  have  more  conversation  or  more 
pilots  in  the  cockpit,  and  when  just  one  would  likely  to  be  safe.  In  addition,  it  needs  to 
tell  us  how  easy  it  is  to  fall  asleep  and  get  good  rest  at  a  given  time. 

I  prefer  to  see  visually  where  sleep  pressure  maximizes  and  where  ambient  light  is  at  a 
minimum  to  decide  when  a  nap  is  possible,  optimal,  and  when  extra  vigilance  is  called 
for. 

Developers  comment:  We  could  provide  a  narrative  report  to  indicate  the  best  times  to 
get  rest  if  time  is  available.  The  software  could  either  print  a  paragraph  providing  advice 
by  day  for  specific  times  or  place  a  short  phrase  at  the  appropriate  time  on  a  timeline 
similar  to  that  available  in  FAST™. 

3.  I  would  like  to  have  multiple  times  across  the  bottom  (sic.  of  the  graph):  Zulu,  Elapsed 
Mission  time,  and  base  time. 

Comment:  He  sent  a  picture  of  what  he  wanted;  we  have  been  unable  to  print  it.  His 
comments  about  it  follow. 

Here’s  a  sketch  of  what  I  wanted  for  C2  software,  I’d  like  something  similar.  You  can 
see  that  I  have  the  amount  of  daylight  fading  in  and  out  in  the  background  across  the 
screen,  and  then  the  local  times  within  that  color  scheme.  Underneath  it,  in  Black  and 
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white,  I  would  have  Zulu  time.  You  could  display  the  elapsed  mission  time  or  elapsed 
alert  time  directly  on  the  Gantt  chart  of  the  mission. 


4.  Can  you  add  a  caffeine  event? 

Few  pilots  use  anything  but  Caffeine  (or  want  to);  concentrate  your  energies  there.  Even 

a  disclaimer  with  approximate  data  would  be  useful. 

5.  Remember  that  for  AMC,  your  tool  needs  to  provide  very  quick  answers  to  make  key 

decisions: 

•  Is  this  mission  safe  as  scheduled? 

•  Does  this  mission  provide  a  minimum  level  of  alertness  assuredness  at  a  critical 
event? 

•  Would  it  be  safer  if  I  shifted  a  ground  time  or  naptime?  (I  want  to  be  able  to  do  this 
graphically  by  dragging  with  my  mouse) 

•  When  is  it  optimal  to  plan  my  work-rest  periods  (when  do  I  order  the  copilot  to  go 
back  and  sleep)? 

•  Such  decision-making  is  routine,  and  takes  place  very  quickly.  Schedulers  and 
controllers  are  task  saturated,  and  they  have  no  time  to  spend  10-20  minutes  building 
in  events  that  cannot  be  easily  shifted.  There  need  to  be  drop  down  menus  for 
standard  sequence  of  events  and  ground  times. 

In  order  to  build  such  a  tool,  a  user  interface  that  allows  rapid  entry  and  easy  changes  is 

needed;  here  is  a  suggested  interface  that  mirrors  GDSS,  our  C2  Tool: 


ICAO  (with 
smart-fill) 

Depart  Time 

Initial  sleep:  Drop  downs  for  (hours,  quality,  end  at: 
alert,  normal  wakeup  time  for  zone,  specific  time) 

ICAO 

Arrive  Time 

(Specify  hard  time  or  approximate  flight  time) 

Depart  Time 

(Hard  time  or  In  sequence  with  drop  down  for 
standard  ground  times) 

Add  ICAO... 

From  this,  the  software  calculates  ground  and  flight  times,  populates  normal  SOP  ground 
and  flight  events  (relative  to  takeoff  times),  and  then  displays  a  Gantt  chart  against  the 
calculated  periods  of  light  and  darkness.  It  calculates  the  perfonnance  and  sleep  pressure 
curves  based  upon  the  single  sleep  period,  and  now  offers  the  following: 

6.  Add  Event  Marker  (Time  from  a  given  event  and/or  Lat/Long,  hard  time  or  in  sequence) 

7.  Add  Sleep  Event  with  point  and  click  (default  relative  to  takeoff  or  landing,  option  for 
hard  time,  length,  quality) 

Then,  any  event  that  is  not  a  hard  time  can  be  slid  left  or  right  with  the  mouse  to  create  a 
recalculation. 
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8.  Grid  view:  the  box  that  opens  at  the  end  of  an  interval  should  be  displaced  to  the  right 
one  box  so  it  does  not  overlap  the  interval.  The  date  on  the  left  should  be  highlighted 
somehow  for  the  row  that  is  currently  being  worked  on.  (I)  Would  like  to  have  a  Save 
button  when  creating  a  long  schedule.  This  would  avoid  the  wait  while  the  schedule  is 
graphed. 

9.  Tabular  view:  would  like  to  have  a  column  with  Lapse  Index  values. 

10.  Graphical  View:  It  would  be  nice  to  be  able  to  “unlock”  the  cursor  position  across  graph 
windows  in  some  cases.  This  is  particularly  true  of  two  schedules  that  have  different  start 
dates  but  overlap  in  time.  There  is  no  way  manually  to  position  the  cursor  at  the  same 
time  in  the  two  schedules. 

These  two  AF  officers  have  provided  excellent  details  for  creating  an  AMC  interface  for 
FAST™  and  F-PAS.  Other  than  the  editing  capabilities  that  were  described,  many  of  the  other 
ideas  were  included  in  the  Final  Phase  2  FAST™  product  (Eddy  &  Hursh,  2006b). 

AMC  Pilots 

As  mentioned  in  the  methods  section,  a  task  analysis  was  not  conducted;  no  task  or  job  appeared 
to  exist  regarding  the  pilots  active  involvement  in  mission  planning.  Furthermore,  the  pilots  felt 
that  the  tool  would  not  be  used  if  it  didn’t  make  a  difference  in  the  schedules  they  were  flying. 
They  stated  that  they  are  given  their  mission  plan  via  TACC’s  Global  Decision  Support  Systern- 
2  (GDSS-2)  and  they  follow  it.  They  believed  that  if  a  mission  needed  changing  and  they  had 
used  our  fatigue  assessment  tool  to  justify  their  schedule  recommendations,  TACC  would  not 
accept  the  change  based  on  a  tool  with  which  they  were  not  familiar.  That  is,  TACC  would 
discount  the  analysis  and  recommendation.  The  pilots  said  they  needed  the  tool  and  would  use  it 
if  TACC  used  it  too. 

The  pilots  further  stated  that  if  TACC  used  the  model  in  their  mission  scheduling,  they  would 
prefer  a  Personal  Data  Assistant  (PDA)  version  of  the  product.  However,  they  stated  that  they 
would  use  a  PC  version  too.  Therefore,  the  discussion  turned  to  several  related  topics  that  helped 
the  development  team  clearly  understand  how  AMC  creates  and  executes  schedules,  and  how  the 
schedules  affect  aircrews.  The  scheduling  process  is  described  below  followed  by  notes  from  the 
development  team. 

Nonnal  Sequence  of  an  AMC  Mission 

1 .  A  multi-sortie,  multi-day  mission  is  created  by  mission  planners  at  TACC.  The  start  date 
and  departure  time  of  the  mission  are  specified.  Regulations  governing  crew  duty  and 
crew  rest  periods  are  considered,  as  are  restrictions  for  the  planned  airfields,  customer 
needs,  and  other  relevant  information.  Presumably,  GDSS-2  is  used  for  mission 
planning.  There  may  be  a  bias  in  mission  planning  toward  using  the  minimum  ground 
times  that  are  consistent  with  regulatory  crew  rest  minima. 

2.  The  mission  plan  is  assigned  to  an  airlift  wing  and  then  to  a  squadron.  The  wing  or 
squadron  may  notice  an  error  or  problem  in  the  plan  and  negotiate  a  change  with  the 
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TACC  mission  planners.  The  wing  or  squadron  assigns  a  crew  and  an  aircraft  to  the 
mission. 

3.  During  the  preflight  planning  period  prior  to  the  first  sortie  of  the  mission,  between  show 
time  and  takeoff  time,  the  assigned  aircrew  acquires  the  mission  plan  in  the  fonn  of 
GDSS-2  files  that  include  timely  weather  and  NOTAM  infonnation  for  the  first  sortie.  If 
there  are  multiple  sorties  in  one  crew  duty  period,  then  the  crew  still  only  receives  the 
files  for  the  first  sortie.  Thus,  after  the  first  sortie  they  must  get  the  flight  plan  for  the 
next  sortie  before  progressing  to  their  eventual  crew-rest  layover.  The  flight  plan  is  filed 
automatically  at  this  time  by  TACC  with  the  FAA  or  ICAO,  probably  using  a  GDSS-2 
function.  The  mission  is  now  under  the  control  of  a  mission-execution  group  at  TACC, 
not  the  original  mission  planners. 

4.  The  takeoff  and  landing  times  for  the  first  sortie  are  acquired  automatically  through  L- 
Band  Satcom,  or  alternatively,  through  an  AMC  Command  Post  and  forwarded  to  TACC. 
Otherwise,  reporting  may  fall  to  UHF  phone  patch,  HF  radio,  or  telephone  after  landing. 

5.  After  landing,  the  crew  debriefs  and  then  enters  crew  rest. 

6.  Steps  3  through  5  are  repeated  for  subsequent  sorties  in  the  mission.  The  GDSS-2  files 
now  include  a  brief  mission  history  of  previous  sorties. 

Mission  Changes 

If  there  are  no  maintenance-,  weather-  or  customer-generated  changes  to  the  planned  mission 
schedule,  the  mission  proceeds  as  planned.  However,  there  will  usually  be  a  change  to  the 
planned  schedule  at  some  point  during  mission  execution.  The  crew  may  leam  of  the  change 
during  a  post-sortie  debrief,  immediately  before  a  show  time,  during  a  preflight  planning  period, 
or  while  in  flight.  If  the  change  takes  the  form  of  a  delayed  takeoff  and  occurs  during  the 
preflight  planning  period,  a  new  GDSS  sortie  plan  is  not  provided  to  the  crew  unless  the  delay 
exceeds  about  four  hours.  Given  that  there  is  a  change,  it  is  now  up  to  the  mission  commander  to 
negotiate  schedule  changes  with  the  mission  execution  group  at  TACC.  If  the  crew  is  notified 
between  the  end  of  crew  rest  and  takeoff  time  of  a  significant  delay  in  takeoff  time,  they  may  re¬ 
start  a  new  crew  rest  period. 

Often,  revised  mission  schedules  are  poorly  devised  by  TACC  because  those  doing  the  revising 
do  not  have  all  the  necessary  information  that  the  original  schedulers  had.  Therefore,  the  aircrew 
is  often  required  to  review  and  correct  the  new  schedule  based  on  their  knowledge  of  airport 
access  time,  ground  support  hours,  etc.  The  new  information  can  sometimes  lead  to  additional 
mission  delay. 

In  theory,  the  aircraft  commander  may  declare  unscheduled  crew  rest  if  the  crew  is  too  fatigued 
to  fly  safely.  This  determination  is  made  through  a  highly-subjective  Operational  Risk 
Management  (ORM)  process  that  may  be  wing-specific  and  is  negotiated  with  TACC.  In  fact, 
an  individual  mission  commander  may  exercise  this  option  only  rarely  (perhaps  once  a  year) 
without  the  threat  of  being  “re-educated”  by  the  wing  or  squadron.  Therefore,  fatigue  must  be 
extremely  severe  for  unscheduled  crew  rest  to  be  declared  by  the  commander. 

27 

Approved  for  public  release;  distribution  unlimited,  Public  Affairs  Case  File  No.  09-402, 

August  21,  2009,  Brooks  City-Base,  Texas. 


One  opportunity  for  input  that  pilots  do  have  is  the  ORM  worksheet  completed  during  Step  3 
above.  Up  to  now,  they  have  had  opportunities  to  input  at  least  subjective  measures  of  fatigue 
into  this  worksheet.  However,  there  seems  to  be  a  strong  culture  of  “pencil- whipping”  these 
sheets  to  make  the  flight  “safe.”  This  culture  has  probably  arisen  due  to  the  entirely,  up  to  now, 
subjective  nature  of  judging  fatigue  levels,  especially  future  fatigue  levels.  In  the  past,  pilots 
have  been  unwilling  to  admit  they  are  too  fatigued  to  fly  because  of  social  pressure  to  not  appear 
vulnerable  to  fatigue. 

The  GDSS-processed  flight  plan  includes  a  great  deal  of  information  that  is  “nice  to  have,”  but 
that  may  not  really  affect  the  actions  of  the  crew.  The  pilots  indicated  that  even  the  vast  majority 
of  weather  infonnation  is  generally  disregarded  unless  it  is  a  severe  weather  situation.  The  team 
did  detect  that  the  aircrew  would  be  generally  interested  in  receiving  a  FAST™  like  plot  along 
with  the  GDSS-2  flight  plan  to  be  referred  to  and  used  as  each  crewmember  saw  fit. 

Our  four  pilots  pointed  out  that  attending  to  and  actively  managing  crew  fatigue  would  be  a 
major  change  in  AMC  operations  and  certainly  not  something  they  saw  occurring  presently. 

They  strongly  indicated  that  the  culture  flies  by  its  regulations  and  after  17-18  hours  of  ground 
time  and  some  crew  rest,  no  matter  what  the  status  of  the  crew  and  whether  or  not  adequate 
recuperative  sleep  was  attained,  the  crew  will  be  required  through  cultural  pressure  to  carry  on. 
This  also  describes  what  members  of  our  task  analysis  team  have  personally  experienced  when 
flying  AMC  missions  off  and  on  over  the  past  30  years. 

Our  pilots  expressed  some  interest  in  a  tool  for  pilots  to  use  on  their  own  (perhaps  PDA  based)  to 
analyze  their  own  fatigue  and  perhaps  make  better  ORM  judgments.  However,  it  was  noted  that 
unless  there  is  buy-in  from  the  command  structure  overseeing  the  ORM  process,  these  “more 
objective”  ORM  analyses  are  likely  to  be  ignored. 

There  was  some  interest  in  a  pilot  tool  for  helping  to  plan  their  personal  schedules  and  adjust  to 
jet  lag  and  phase  shifting.  It  was  expressed  that  these  data  need  to  be  kept  anonymous.  It  was 
suggested  that  our  FAST™-like  tool  should  really  calculate  the  schedule’s  impact  on 
perfonnance  based  on  the  overall  crew  and  assume  that  the  cognitive  perfonnance  rating  of  the 
individual  is  really  an  average  for  the  entire  crew. 

There  was  some  indication  that  pilots  might  be  willing  to  wear  wrist  activity  monitors  for 
keeping  track  of  their  own  sleep/wake  data.  However,  the  upload  system  would  have  to  be 
simple.  It  was  expressed  that  PDAs  are  really  the  only  technology  that  people  are  likely  to  use 
on  a  regular  basis,  as  most  are  unlikely  to  pull  out  a  laptop  to  evaluate  the  impact  of  their  sleep 
schedule.  The  pilots  indicated  that  the  PDA  is  the  only  format  that  an  aircraft  commander  would 
in  reality  have  the  time,  and  take  the  time,  to  actually  update  a  FAST™-like  schedule  as  a 
mission  progressed  from  point  to  point. 

It  was  suggested  that  the  schedule  infonnation  might  be  best  recorded  in  an  Excel  fonnat  as  it 
can  be  viewed  on  a  PDA  and  is  easily  transferable  via  a  thumb  drive  and  one  can  simply  click 
through  cells  with  one  of  the  directional  buttons  to  detennine  cognitive  effectiveness  at  any 
given  point  along  the  schedule.  (PDAs  now  accept  thumb  drives.) 
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A  display  format  was  suggested  that  superimposed  a  performance  effectiveness  curve  on  a 
mission  laid  out  horizontally  across  the  screen.  Each  part  of  the  mission  was  labeled  such  as  pre¬ 
mission,  KSKF-KSUN,  ground  time  (with  sleep  time),  etc.  It  was  not  clear  how  this  improved 
on  a  FAST™  graph  with  labels  for  the  intervals.  However,  it  was  clear  that  the  pilots  preferred  a 
timeline  display  where  they  could  clearly  see  trends  in  the  perfonnance  effectiveness  curve  as 
the  mission  proceeded.  One  of  our  team  members  suggested  that  our  mission  timeline  be 
modified  to  highlight  flight  time  versus  ground  time  instead  of  highlighting  every  other  four- 
hour  block  of  time. 

The  433  OG/CCM  possessed  electronic  files  of  the  flight  plans  actually  flown  by  the  wing  over 
several  months.  Evaluation  of  these  schedules  with  our  model  and  ordering  them  with  respect  to 
their  fatigue  impact  on  the  aircrew  would  be  supportive  of  AMC/CC’s  intention  to  begin 
evaluating  and  doing  something  about  excessively  fatiguing  AMC  missions.  If  change  is  in  the 
wind  at  AMC,  it  may  be  that  a  FAST™-like  tool  could  contribute  to  it  as  we  attempt,  perhaps,  to 
integrate  it  with  GDSS  2. 

Potential  Counter-Fatigue  Intervention  Recommendations 

First,  the  SAFTE™  model  may  be  used  by  GDSS-2  during  mission  planning  to  optimize  the 
mission  schedule  to  minimize  fatigue.  The  model  would  need  to  be  provided  with  acceptable 
maxima  and  minima  for  takeoff  and  landing  times  (per  individual  airfield  restrictions),  expected 
sortie  lengths,  and  lat/lon  data.  If  AMC  chooses  not  to  integrate  the  SAFTE™  model  with 
GDSS-2,  the  local  Wing  schedulers  could  still  use  a  FAST™-like  tool  for  generating  local 
mission  flight  plans. 

Second,  the  SAFTE™  model  may  be  used  passively  by  GDSS-2  during  mission  planning  to 
calculate  and  report  critical  in-flight  fatigue  periods  in  an  ORM  context  and  recommend  timely 
and  useful  fatigue  countermeasures  (Miller  &  Eddy,  2009).  The  GDSS-2  generated  package 
acquired  by  the  aircrew  at  step  3,  above  would  include  this  output.  Problem:  if  there  is  a  several- 
hour  delay  in  takeoff  time  and  no  new  GDSS-2  files  and  fatigue  countenneasures 
recommendations  are  generated,  then  the  original  countermeasure  recommendations  may  no 
longer  be  appropriate. 

Third,  a  personal  version  of  F-PAS  may  be  used  by  a  mission  commander  to  track  his/her  own 
fatigue  level  during  a  mission.  The  GDSS-planned  mission  schedule  could  be  downloaded  to  the 
personal  device  before  the  first  sortie.  A  wrist  activity  monitor  could  be  used  by  aircrew  enroute 
to  download  actual  sleep  times  to  the  personal  device/s.  Sleep  times  could  also  be  input  from 
activity  logs  maintained  by  individual  aircrew.  The  output  of  this  “personal  system”  could  feed 
quantitatively  into  the  ORM  process  such  that  it  aids  the  mission  commander’s  crew  rest 
negotiations  with  TACC.  Anonymity  of  the  data  from  aircrew  would  be  required  if  results  were 
transmitted  to  TACC.  The  record  of  events  and  work  interrupting  adequate  sleep  could  also  be 
used  to  support  the  commander’s  decisions  during  the  “re-education”  process  that  might  follow. 

The  pilots  provided  information  regarding  how  mission  schedules  are  received  by  the  aircrew. 
This  is  accomplished  via  a  flight-planning  computer.  Apparently,  the  flight-planning  computer 
is  a  stand-alone  machine.  Assuming  F-PAS  is  running  in  a  stand-alone  mode,  it  could  receive 
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the  schedule  data  from  the  download  of  the  route-of- flight  attachment  or  from  the  flight-planning 
computer.  The  fonnat  of  the  data  coming  from  either  place  would  have  to  be  standard  and  the  F- 
PAS  software  would  have  be  able  to  read  it  for  the  information  it  would  need,  flight  times, 
waypoints,  etc. 

With  the  above  information  and  the  morning  session  ending,  we  decided  we  should  not  impose 
on  the  pilots  for  additional  time.  Therefore,  a  full  TA  was  not  completed;  we  did  not  run  through 
a  scenario  on  how  they  would  use  the  product  or  where  it  would  fit  into  their  mission  planning, 
execution,  or  re -planning.  We  discussed  general  concepts  that  indicated  they  would  need  to  have 
extensive  schedule  editing  capability,  and  be  able  to  answer  “what-if  ’  questions  comparing 
alternative  schedules.  With  these  concepts  in  mind,  we  presented  some  design  specifications  for 
a  PDA  version  of  F-PAS  in  the  final  report  of  the  contract  (Eddy,  Miller  &  Moise,  2008). 

Flight  Surgeons 

Through  the  discussion  with  the  FS,  we  learned  whom  the  FS  envisioned  as  the  primary  users  of 
the  fatigue  countermeasures  interface.  They  believed  that  the  phannaceutical  fatigue 
countenneasures  option  in  the  tool  should  be  available  to  all  aircrew  members.  They  did  not 
want  to  be  in  a  position  of  pushing  Go  and  No-Go  pills  on  the  aircrew  for  fatiguing  missions, 
viewing  such  a  practice  as  being  both  unethical  and  in  opposition  to  AF  regulations.  They 
wanted  the  aircrew  to  use  the  modeling  tool  to  see  for  themselves  the  impact  of  fatigue  on  their 
predicted  performance,  experiment  with  the  Go-No  Go  pill  options,  see  the  possible  benefits  to 
their  performance,  and  then  go  to  their  FS  for  advice  on  administration.  As  the  FS  saw  it,  the  FS 
could  then  prescribe  the  appropriate  medication  and  medication  schedule  for  the  aircrew  based 
on  the  mission  schedule,  time  of  day,  and  post  mission  activities.  Together,  the  FS  and  aircrew 
would  use  F-PAS  as  a  decision  aid  to  help  them  detennine  the  best  countermeasure  with  the 
fewest  side  effects.  Through  the  ensuing  discussion,  the  FS  provided  answers  to  the  following 
questions. 

Who  would  the  product  users  be,  who  should  be  included  and  who  should  be  excluded? 

The  FS  insisted  that  the  users  would  be  aircrew  and  commanders,  as  well  as  FS,  and  that  the 
interface  should  be  designed  to  accommodate  aircrew  scheduling  goals  and  tasks,  not  those  of 
the  FS.  Therefore,  in  answer  to  the  question  of  who  the  users  might  be,  it  was  concluded  that  no 
special  interface  was  needed  for  FS  to  access  the  model  and  obtain  performance  predictions.  We 
concluded  that  we  should  follow  the  user  requirements  for  pilots  and  aircrew. 

Generally,  what  knowledge,  skills  and  abilities  did  users  need  to  use  pharmaceutical  fatigue 
countermeasures  options? 

Since  the  interface  would  be  used  by  aircrew,  standard  computer  use  skills  would  be  needed 
along  with  an  understanding  of  flight  scheduling  events  and  times.  Since  most  aircrew  are 
computer  literate  and  intimately  familiar  with  crew  scheduling  practices,  their  general  knowledge 
and  skills  should  be  adequate  to  navigate  the  browser-based  interface  for  access  to  the  tool. 

What  special  skills/abilities  do  users  need  to  use  pharmaceutical  fatigue  countermeasures 
options? 
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Some  context  sensitive  help  would  be  needed  to  present  how  Go-No  Go  pills  worked,  their  time 
course,  interactions  with  other  substances,  and  side  effects.  This  infonnation  could  be  created  by 
the  RAMs  for  final  inclusion  in  the  browser-based  interface  for  projecting  the  effects  of 
phannaceutical  fatigue  countermeasures. 

Other  comments  by  the  FS  helped  to  reinforce  our  existing  interface  ideas  developed  with  the 
schedulers  and  emphasized  the  importance  and  priority  of  specific  interface  features.  The  FS 
ideas  included: 

•  A  graphic  display  similar  to  FAST™  with  nonnal  (un-medicated)  perfonnance  and 
medicated  perfonnance  on  the  same  graph. 

•  A  sleep  propensity  graph  comparing  the  two  conditions. 

•  How  FS  received  the  flight  schedule.  They  listed: 
o  Attending  squadron  meetings 

o  Aircrew  walk-in 

•  How  the  flight  schedule  was  fonnatted  when  FS  received  it: 
o  Mission  print  out 

o  Pilot  Flight  Planning  System  (PFPS)  printout 

o  Sometimes  the  safety  officer  or  an  aerospace  physiologist  brings  them  a  FAST™ 
graph  with  concern  for  fatigue  risk. 

•  One  RAM  with  extensive  operational  experience  indicated  that  we  should  explore  the 
PFPS,  the  tool  that  aircrews  use  to  plan  their  training  missions,  especially  in  AFSOC.  If 
our  product  could  read  the  mission  information  from  PFPS,  it  would  reduce  the  data  entry 
time  for  the  fatigue  analysis  and  reduce  data  entry  errors.  They  believed  that  we  could 
leam  about  PFPS  with  navigator  training  squadrons  that  fly  the  T-43  aircraft.  We  may  be 
able  to  attend  a  monthly  PFPS  class  there. 

•  Have  web-based,  fatigue  management  training.  Although  this  was  not  within  the 
statement  of  work  of  our  existing  contract,  this  would  be  an  excellent  task  to  add. 

•  F-PAS  should  have  templates  (they  didn’t  say  what  kind)  that  were  aircraft-type  specific. 
That  way  the  aircraft  type,  elicited  from  an  initial  question,  could  determine  some  of  the 
follow-on  questions  and  even  determine  the  format  for  data  entry.  The  most  specific 
aircraft  type  infonnation  includes  show  time  (the  number  of  hours  before  take-off  that 
must  be  scheduled)  and  the  Go/No-Go  pill  regulations. 

•  Create  a  kneepad  version  of  the  output  for  pilots. 

•  Have  links  to  MAJCOM  policy  letters  on  the  use  of  Go  and  No-Go  pills  in  AF  aviation. 

•  On  the  use  of  caffeine  as  a  fatigue  countermeasure,  the  FS  were  concerned  about  making 
predictions  for  caffeine  dependent  aircrew.  From  our  discussion,  we  decided  that  it 
would  be  important  to  know  the  caffeine  dependency  of  the  subjects  used  in  the  Walter 
Reed  caffeine  study,  upon  which  the  cognitive  perfonnance  predictions  were  modeled. 

In  answer  to  this  question  the  following  quote  was  taken  from  the  publication 
documenting  the  study:  “All  subjects  selected  for  participation  reported  . .  .daily  caffeine 
consumption  of  <400  mg”  (Wesensten,  Killgore,  &  Balkin,  2005).  Four  hundred  mg  of 
caffeine  is  approximately  3-5  cups  of  coffee  (http://coffeefaq.com/caffaq.html). 

•  For  educational  purposes,  include  aircraft-type-specific  intervention  examples  in  a 
PowerPoint  fonnat. 

•  Make  each  drug  an  event  that  can  be  used  in  any  of  the  F-PAS  interfaces,  as  an 
educational  tool. 
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•  By  contract,  create  a  USAF  login  at  a  website  outside  of  the  military  website  (.MIL). 

That  way,  any  aircrew  can  log  in  from  any  location  in  the  world.  If  the  tool  is  only  on  the 
AF  website,  crew  and  FS  could  only  log  in  from  a  .MIL  computer. 

WALK-THROUGH 

FAST™  was  originally  designed  for  mission  planning  and  flight  schedule  evaluation.  As  such, 
the  requirements  for  the  Mission  Interface  were  essentially  the  same  for  the  mission  interface  as 
for  FAST™.  With  the  extensive  data  gleaned  through  a  usability  evaluation  of  FAST™ 
conducted  with  the  AMC  schedulers,  a  walk-through  evaluation  was  deemed  unnecessary.  The 
excellent  comments  from  the  SMEs  regarding  FAST™  displays,  data  entry,  and  editing  approach 
was  judged  sufficient  for  designing  a  new  interface  built  on  SME  advice  and  web  programming 
constraints.  Once  a  rudimentary  Mission  Interface  was  complete,  a  group  of  potential  SMEs  was 
assembled  for  an  inspection  evaluation. 

INSPECTION  EVALUATION 

A  Travis-to-Ramstein  scenario  (Appendix  C)  was  used  for  an  inspection  evaluation,  which 
occurred  in  the  computer  laboratory  of  the  altitude  training  group  of  the  USAF  School  of 
Aerospace  Medicine  (USAFSAM/ATA),  Brooks  City-Base,  Texas,  on  the  afternoon  of  25 
September  2008.  The  SMEs  were  three  flight  surgeons  from  the  USAFSAM  Residency  in 
Aerospace  Medicine  (RAM)  program,  a  program  that  teaches  fatigue  management.  All  three 
were  Majors  with  7,  8,  and  14  years  of  active  duty.  One  had  used  FAST™  for  mishap  analysis, 
one  had  been  exposed  to  FAST™,  and  one  was  completely  unfamiliar  with  FAST™.  One  pair 
of  SMEs  walked  through  the  scenario,  and  then  a  single  SME  walked  through  the  scenario,  with 
one  observer  taking  notes.  The  session  with  the  single  SME  produced  a  representative  elapsed 
time  for  data  entry  and  analysis  of  the  scenario  for  a  novice  user:  33  minutes. 

No  significant  errors  and  only  two  minor  reversals  occurred  during  data  entry  and  display  for  any 
of  the  SMEs.  Twelve  assists  were  logged  on  the  data  form  (Appendix  B)  across  the  three  SME 
sessions. 


Comments  from  Assists 

•  Two  of  the  SMEs  were  confused  by  the  right-hand  placement  of  the  window  for  city  and 
base  location  data  entry  on  the  Edit  Properties  page.  They  expected  a  drop-down  list  to 
occur  within  the  window  in  which  they  were  working.  One  SME  also  missed  the  “Use 
This”  button. 

•  One  SME  noted  the  absence  on  the  Edit  Schedule  page  of  instructions  to  select  the  Edit 
Schedule  option  after  executing  the  Edit  Properties  option.  Later,  the  absence  on  the 
same  page  was  noted  for  an  instruction  to  use  the  Effectiveness  Graph  option  after  using 
the  Edit  Properties  and  Edit  Schedule  options.  The  expected  sequence  of  use  of  the 
options  should  be  described  on  the  Edit  Schedule  page. 
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•  Two  of  the  SMEs  needed  assistance  to  click  within  the  calendar  to  obtain  the  pop-up 
window  that  initiates  the  entry  of  work  and  sleep  periods  and  events.  A  where -to-click 
instruction  near  the  calendar  is  needed. 

•  One  SME  needed  assistance  to  use  the  up-down  arrow  functions  for  adjusting  date  and 
time  for  sleep  and  work  periods  and  events. 

•  All  of  the  SMEs  needed  assistance  when  trying  to  enter  the  first  event.  They  had  been 
instructed  to  enter  work  periods  before  events.  Intuitively,  they  clicked  on  existing  work 
periods  in  the  schedule  for  the  entry  of  an  event  associated  with  the  work  period.  In  fact, 
they  needed  to  click  on  empty  space  in  the  schedule  to  initiate  data  entry.  Two 
suggestions  followed.  First,  put  an  instruction  on  the  page.  It  should  be  located  with  the 
instruction  suggested  in  the  preceding  paragraph.  Second,  default  the  work  period 
indicator  to  a  narrower  column  in  the  calendar  so  that  white  space  is  available  next  to  it. 

•  One  SME  failed  to  see  the  event  type  field. 

•  One  SME  requested  an  explanation  of  why  different  events  have  different  heights  when 
displayed  in  the  calendar,  and  why  an  event  occupies  30  minutes  in  the  calendar  display 
when  it  actually  occupies  only  a  moment  in  time.  An  event  should  be  defaulted  to  one 
minute. 

•  One  SME  was  confused  by  event  color-coding.  Perhaps  an  explanation  in  the  help  file 
would  suffice. 

Reversals 

Two  SMEs  experienced  a  software  problem.  One  SME  had  committed  a  minor  error  in  data 
entry:  he  entered  a  sleep-period  end-time  earlier  than  the  start  time  (he  failed  to  change  the  date 
on  an  over-midnight  period).  A  debugger  error  message  occurred,  from  which  recovery  ensued 
by  using  the  browser  “back”  function.  However,  that  sleep  period  had  acquired  the  start  and  end 
times  of  the  preceding  work  period.  The  two  periods  appeared  next  to  each  other  in  the  calendar, 
both  labeled  “Work,”  with  one  colored  orange  and  one  colored  blue.  The  same  display  anomaly 
occurred  for  the  other  SME,  but  whatever  error  he  had  committed  was  too  obscure  to  recall.  The 
software  problem  caused  minor  reversals  for  the  SMEs. 

Additional  Comments 

•  Reduce  the  empty  space  on  the  Main  Menu  page  to  prevent  the  need  to  scroll  vertically. 

•  Change  the  cursor  shape  when  hovering  over  a  hot  link  on  the  Main  Menu  page. 

•  There  was  confusion  about  the  Save  and  Exit  button  on  the  Edit  Properties  and  Visual 
Schedule  Editor  pages.  What  was  saved?  Where?  How  did  this  function  differ  from  the 
Save  File  function  on  the  Edit  Schedule  page? 

•  The  default  location  for  a  given  event  should  echo  the  most  recent  location  known  in  the 
schedule. 

•  To  edit  a  sleep  period,  work  period  or  event,  a  right  mouse  click  would  be  preferable  to  a 
double-left  click. 

•  When  a  sleep  or  work  period  is  being  entered,  the  default  length  should  be  greater  than 
30  minutes.  (Three  hours  might  be  useful.) 

•  The  relationship  between  “Location”  and  “Location  Select”  fields  needs  better  functional 
grouping. 
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•  The  nature  of  the  Description  field  (i.e.,  a  non-required  comments  field)  was  unclear. 

•  When  events  were  entered  during  a  work  period  after  midnight,  and  the  work  period 
extended  over  midnight,  the  work  period  filled  the  whole  calendar  column  before 
midnight  and  only  half  the  column  after  midnight.  This  was  a  bit  confusing  in  tenns  of 
continuity  across  midnight. 

•  As  an  alternative  to  clicking  within  the  calendar,  place  Sleep,  Work  and  Event  buttons 
above  the  top  left  of  the  calendar. 

•  The  hover  window  over  an  event  should  not  indicate  that  the  event  is  30  minutes  long. 

•  The  6-hour  and  1 -minute  resolutions  on  the  graph  may  not  be  useful.  Two-  and  3-hour 
resolutions  may  be  useful. 

•  The  sleep  and  work  period  indicators  along  the  x-axis  of  the  graph  should  be  on  separate 
lines.  This  would  allow  overlap  to  be  visible. 

•  Need  cut-and-paste  tabular  reports,  as  used  in  FAST. 

•  The  items  in  the  Dashboard  need  better  functional  grouping,  such  that  numeric  data  are 
related  to  flags  more  clearly. 

•  Do  not  allow  users  to  change  the  boundaries  of  the  green  and  yellow  areas  on  the  graph. 
Changing  the  level  of  the  criterion  line  is  good. 

•  During  file  saving  operations,  instruct  the  user  to  avoid  adding  a  filename  extension. 

•  The  sequence  of  operations  on  the  file  saving  screen  was  not  intuitive. 

•  Overall,  the  windows  used  are  too  tall  for  use  on  Department  of  Defense  visual  displays. 
All  DoD  computers  have  a  security  function  that  places  a  blue-colored  status  bar  across 
the  top  of  the  screen.  It  is  about  the  height  of  the  Windows  menu  bar  at  the  bottom  of  the 
screen. 

Questionnaire  Responses 

The  following  three  questions  were  evaluated  by  the  SMEs  with  subjective  ratings,  using  the 
following  scale: 

•  Very  acceptable  (1) 

•  Acceptable  (2) 

•  Borderline  (3) 

•  Unacceptable  (4) 

•  Very  unacceptable  (5) 

Please  rate  the  ease  of  application  of  the  interface  to  the  intended  task:  the  simplicity  with  which  the 
interface  can  be  employed  to  determine  whether  fatigue  was  a  factor  in  a  mishap.  In  an  ideal  world,  the 
interface  would  be  totally  natural  and  predictable  in  behavior.  Nothing  should  obstruct  your  progress  in 
completing  this  task. 

•  The  ratings  of  all  three  SMEs  were  “Acceptable  (2).” 

Rate  the  performance  of  the  interface:  the  speed  with  which  the  interface  responds  to  requests. 

Rate  the  support  information  for  the  interface:  the  information  available  to  acquire,  use  and  support  the 
interface.  Encompasses  initial  instructions,  user  guides,  tutorials,  integrated  assistance,  context  sensitive 
help. 

•  The  rating  of  one  SME  was  “Very  acceptable  (1)”  and  the  other  two  were  “Acceptable 
(2).” 
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Rate  the  interface ’s  function:  the  overall  capabilities  of  the  interface. 

•  The  ratings  of  all  three  SMEs  were  “Acceptable  (2).” 

The  following  open-ended  questions  generated  several  comments  from  the  SMEs. 

What  were  your  objectives  as  you  tested  this  interface? 

•  “[Assess]  usability  for  a  novice  [user]”  (one  response) 

Was  the  scope  of  the  usability  testing  that  you  did  adequate  to  meet  your  objectives? 

•  “Yes”  (one  response) 

Can  you  suggest  another  method  of  raw  data  entry  that  would  reduce  time,  prevent  entry  errors,  and 
provide  greater  awareness  of  data  conflicts/errors? 

•  “More  point  and  click  and  pop-ups” 

•  “Use  the  [calendar]  grid  with  right  click  capability  [for  changes]  and  15-minute  intervals” 

•  “[The]  sleep  interval  default  time  should  be  set  to  8  hours,  not  30  minutes” 

•  “Allow  click  on  work  [period]  to  [enter]  events” 

Can  you  suggest  other  data  editing  methods  that  would  work  on  a  web  page  and  would  be  more  powerful 
for  making  changes? 

•  “Right  click  with  pull-down  [menu]” 

•  “Pop-up  locations  should  start  with  last  known  location,  i.e.,  take-off  location  should 
[default  to]  last  land  location” 

Could  the  interface  graph  be  formatted  differently  to  better  assist  you  in  completing  your  mishap 
investigation  and  report? 

•  “Separate  work  and  sleep  bars  on  the  bottom  [on  the  x  axis]” 

What  other  improvements  should  be  made  to  the  interface? 

•  “More  leading  questions” 

•  “More  click  and  point” 
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SUMMARY  OF  BETA  INTERFACE 


After  signing  into  the  F-PAS  website  with  user  identification  and  a  password,  the  user  may  select 
from  the  Mission  Scheduler  options  to  create  a  new  schedule,  open  an  existing  schedule,  or 
import  a  schedule  from  FAST™.  If  a  new  or  previous  schedule  is  selected,  the  user  may  enter  or 
edit  schedule  data.  Starting  with  the  schedule  properties  the  user  names  the  schedule,  may  enter 
a  short  description,  gives  it  a  starting  location,  specifies  the  time  (Zulu  or  local),  and  then  saves 
all  entries  and  exits  the  Properties  window.  Upon  return  to  the  Edit  Schedule  (high  level) 
window,  the  user  may  enter  their  schedule.  In  the  Visual  Schedule  Editor,  the  user  enters  objects 
(elements)  such  as  sleep,  work,  and  events  (takeoff,  landing,  etc.).  These  entries  are  all  created 
using  a  calendar  like  display  with  the  days  across  and  the  hours  down  the  page.  Figure  3  shows 
the  data  entry  screen  for  a  week’s  worth  of  days.  A  user  can  easily  move  between  different  days, 
months,  or  years  editing  sleep  intervals,  work  intervals,  and  events.  Figure  3  also  shows  how  a 
user  can  switch  between  local  and  Zulu  time  bases  for  data  entry  or  editing  as  needed. 
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Figure  3.  Mission  interface  calendar-layout,  data-entry  screen. 

Once  an  object  is  entered  it  can  be  moved,  lengthened,  or  shortened  by  click  and  drag.  A  pop-up 
window,  activated  by  clicking  on  the  interval  or  event,  allows  editing  to  the  minute.  There  are 
preprogrammed  events  such  as  take-off,  landing,  drug  events,  etc.  Each  may  be  given  a  location 
by  selecting  the  nearest  city,  base,  or  inserting  lat/lon.  Events  with  lat/lon  coordinates  allow 
F-PAS  to  track  the  light  conditions  on  the  earth.  Returning  to  the  Edit  Schedule  (high  level) 
menu  the  user  may  generate  a  graph  of  performance  effectiveness  shown  in  Figure  4  by  clicking 
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on  Effectiveness  Graph.  The  graph  can  be  zoomed  to  several  resolutions  ranging  from  1  minute 
to  1  day.  Figure  4  shows  a  graph  at  6-hour  resolution  on  a  Zulu  time  base.  The  infonnation  in 
the  graph  may  be  customized  to  change  the  color  zone  limits,  criterion  line  level,  and  right  axis 
scales.  The  green  zone  on  the  graph  (default  100%  to  90%)  is  the  range  of  performance  during  a 
normal  daytime  duty  day  following  an  eight-hour  period  of  excellent  sleep  at  night.  The  yellow 
zone  (default  90%  to  65%)  is  the  range  of  performance  during  the  24  hr  period  after  missing  one 
night  of  sleep.  The  criterion  line  divides  the  Yellow  Zone  in  the  middle  (default  77.5%)  and  is  a 
guide  for  using  countermeasures  to  enhance  performance.  Performance  in  the  yellow  zone 
below  the  criterion  line  (BCL)  represents  the  performance  of  a  person  during  the  day  following 
loss  of  an  entire  night’s  sleep.  The  red  zone  (default  below  65%)  indicates  performance  that  is 
below  the  level  that  is  acceptable  for  operations.  The  red  zone  represents  performance  following 
sleep  deprivation  of  two  full  days  and  a  night.  Reaction  time  in  the  Red  Zone  is  more  than  50% 
longer  than  that  of  a  well-rested  person. 


Figure  4.  This  image  was  taken  from  the  F-PAS  graphical  output  screen.  The 
graph  shows  performance  at  a  6-hour  resolution  on  a  Zulu  time  base. 

Summary  tables  of  average  performance  effectiveness  for  each  awake,  work,  and  sleep  period, 
and  each  event  may  be  seen  by  clicking  on  Summary  Data  Tables  at  the  top  of  the  main  screen. 
The  tables  also  include  in  the  bottom  row,  average  effectiveness  and  the  percent  of  time  BCL. 

The  user  may  save  the  schedule  to  the  server  at  any  time.  Using  the  Managing  Files  button  on 
the  main  menu,  files  may  be  transferred  to  a  local  computer. 
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Managing  Files.  From  the  Main  Menu,  a  user  may  select  Manage  Files  to  perform  file 
management  functions.  The  user  may  move  files  from  the  server  to  a  local  computer  or  vice 
versa,  rename  files,  or  delete  files.  The  user  selects  the  function,  then  selects  the  file/s  and  then 
presses  the  button  to  make  it  happen.  The  mission  and  mishap  files  are  listed  in  one  column  with 
the  shift  work  files  in  another.  Mission  and  mishap  schedules  may  be  viewed,  edited,  or 
displayed  using  either  of  the  two  interfaces.  The  shift- work  schedule  files  are  only  available 
using  the  shift  work  interface.  Files  left  on  the  server  are  secure  since  each  user  has  their  own 
password-protected  space  and  all  files  are  encrypted. 

Planned  Software  Changes 

Some  of  the  features  planned  for  the  mission  interface  were  not  available  for  testing  during 
inspection  evaluation.  These  features  are  briefly  described  here  along  with  other  changes  noted 
during  the  evaluation.  The  planned  additions  are  followed  by  a  list  of  possible  future  additions. 

Planned  additions 

•  Phannaceutical  fatigue  countermeasures  -  This  important  feature  for  AF  aviation 
involves  inserting  pharmaceutical  event  into  the  schedule  and  observing  its  effect  of 
perfonnance  along  side  unassisted  performance.  It  also  involves  a  plot  showing  the 
effect  of  the  drug  on  sleep  compared  without  the  drug. 

•  Transmeridian  Autosleep  -  This  tool  will  allow  a  scheduler  to  insert  sleep  automatically 
into  the  schedule  based  on  work  times.  It  does  this  using  a  model  of  when  people  sleep 
given  their  work  schedule  and  goals.  The  model  is  based  on  whether  the  individual  is  a 
tourist  or  wants  to  stay  on  their  point  of  origin  schedule.  Aircrew  fall  between  the  two 
extremes  and  rules  are  included  for  such  objectives. 

•  ORM  Report  -  This  will  be  a  narrative  report  that  rates  the  overall  schedule,  the  work 
interval,  and  the  events  with  respect  to  fatigue  ORM.  It  will  identify  fatigue  points  in  a 
schedule  and  recommend  remediation. 

•  Mission  timeline  -  Similar  to  the  mission  timeline  option  in  FAST™,  a  temporally 
organized  table  showing  in  30  minute  or  1  hour  blocks  the  progress  of  a  mission 
including  a  column  for  perfonnance  effectiveness,  illumination,  scheduled  events,  and 
times  to  nap. 

•  Copy  and  paste  -  The  addition  of  copy  and  paste  to  the  clipboard  of  all  screens,  tables, 
and  pop-ups. 

•  Simultaneous  display  of  schedule  and  graph. 

•  Grouping  of  schedule  elements  for  simultaneous  moves  and  copying. 

Potential  future  additions 

•  Assessment  of  fatigue  for  multiple  crewmembers  (different  sleep  history)  on  the  same 
mission  (minor  variation  is  schedules). 

•  Use  of  shape  coding  for  different  types  of  events:  T/O,  AR,  CE,  TD,  and  pre-  and  post¬ 
flight  activities. 

•  A  function  that  maximizes  performance  for  a  particular  time  point. 

•  Stand  alone  and  PDA  versions  of  F-PAS. 

•  Both  Zulu  and  local  time  changing  with  location  on  the  x-axis  simultaneously. 
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DISCUSSION 


AMC  has  been  flying  more  and  longer  missions  with  fewer  pilots  and  fatigue  has  become  a 
safety  issue.  A  fatigue  assessment  and  management  tool  built  on  a  scientifically  based  model  of 
sleep  and  cognitive  performance  can  be  helpful  in  managing  the  fatigue  problem.  By  the 
summer  of  2006,  the  aerospace  physiologists  of  the  B-2  community  had  used  FAST™ 
successfully  to  plan  more  than  2,000  hours  of  long-duration  missions  (personal  communication, 
2006).  The  general  plan  for  designing  the  Fatigue -Performance  Assessment  System  (F-PAS) 
interfaces  was  to  use  two  main  conceptual  pathways  for  scheduling:  regular  schedules  and 
irregular  schedules.  Regular  schedules  for  shift  workers  would  use  the  sequential,  prescriptive 
approach  documented  in  Miller  (2006).  The  Shiftwork  Scheduling  Interface  described  in  Miller, 
Eddy,  Smith,  &  Moise  (2009)  supports  this  approach.  Schedulers  creating  and  evaluating 
irregular  schedules  for  long-aviation  or  sustained  operations  missions  would  use  an  interface 
similar  to  existing  FAST™  methods.  This  “descriptive”  approach  allows  the  user  to  generate  a 
schedule  without  any  constraints  and  the  model  evaluates  it  for  fatigue  effects.  After  collecting 
requirements  from  several  AF  user  groups,  two  interfaces  for  irregular  schedules  were  designed, 
one  for  mishap  investigation  and  one  for  mission  planning  that  would  be  used  by  schedulers, 
aircrew,  and  flight  surgeons.  The  task  analysis  and  usability  assessments  described  in  the  results 
section  were  the  basis  for  designing  the  Mission  Scheduler  Interface.  The  Mishap  Investigator 
Interface  is  described  in  Eddy,  Miller,  &  Smith  (2009). 

From  the  schedulers’  TA  and  remarks,  the  interface  required  quick  data  entry  if  schedules  could 
not  be  electronically  imported  from  some  other  source.  A  graphical  depiction  of  the  schedule 
that  conveys  the  big  picture  was  required  to  maintain  situation  awareness  of  the  mission  goals 
and  objectives.  Graphical  output  was  needed  to  support  quick  comprehension  of  the  schedule 
impacts  on  fatigue  and  performance.  Schedule  editing  was  needed  to  support  moving  sleep, 
work,  and  event  objects  quickly  and  easily  in  the  mission  schedule.  Ease  of  editing  was  deemed 
necessary  for  both  original  schedule  planning  and  for  re -planning  by  aircrew  in  the  field  to 
achieve  optimal  performance  and  productivity  while  maintaining  safety  of  flight  and  operation. 
The  tool  was  to  support  the  grouping  of  sleep,  work,  and  event  objects  to  maintain  situation 
awareness  and  speed  of  answering  “what-if  ’  type  questions  related  to  fatigue.  In  short,  an 
enhanced  interface  that  better  supported  FAST™  functionality  was  required  for  AMC  mission 
schedulers  and  aircrews.  The  AMC  pilots  indicated  that  they  would  be  comfortable  with  a 
similar  interface  and  would  use  the  fatigue  modeling  tool  if  TACC  also  used  it.  Their  most 
preferred  format  would  be  for  the  tool  to  run  on  a  PDA. 

In  our  requirements  assessment  meeting  with  FS,  we  learned  that  they  did  not  want  a  special 
interface  for  themselves.  They  wanted  the  aircrew  to  have  the  option  of  trying  various  stimulant 
and  sleep  aid  alternatives  with  our  fatigue  analysis  tool  when  they  could  not  acquire  sufficient 
sleep  prior  to  a  mission  to  perform  it  without  excessive  fatigue.  The  FS  did  not  want  to  be  in  the 
position  of  pushing  these  medications  on  the  aircrew;  they  wanted  the  aircrews  initiating  a 
request  for  the  prescription.  This  approach  would  allow  the  FS  an  opportunity  to  discuss  pros 
and  cons,  alternatives,  side  effects,  dose  timing,  etc.  with  the  aircrew.  The  FS  did  want  to  review 
the  fatigue  countermeasures  interface  to  insure  that  it  presented  the  alternatives  accurately  and 
that  we  provided  sufficient  infonnation  on  the  side  effects  of  the  medications,  such  as  stimulants 
preventing  sleep,  etc. 
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Even  though  the  first  meeting  was  short,  did  not  include  a  TA  of  FS  tasks  in  making  decisions  on 
Go-No  Go  pill  administration,  and  we  had  several  digressions,  we  learned  much  about  what  FS 
needed  and  did  not  need  in  our  tool.  The  information  in  the  Results  section,  above,  defines  the 
requirements  for  the  fatigue  countenneasures  interface  adequately.  The  big  take-away  message 
from  the  session  was  that  the  fatigue  countermeasures  interface  should  be  designed  for  the 
aircrew  to  allow  them  to  explore  options  for  fatigue  management  during  long-duration  and 
nighttime  missions.  Further,  that  the  interface  be  so  easy  to  use  that  “even  a  caveman  can  do  it.” 
With  that  said,  once  aircrew  detennined  that  opportunities  for  adequate  sleep  could  not  be  used 
to  minimize  fatigue,  they  could  work  jointly  with  FS  on  the  best  countenneasures  to  use  and  the 
best  time  before  or  during  the  mission  to  use  them.  Although  the  pharmaceutical  fatigue 
countenneasures  interface  was  not  complete  at  the  time  of  usability  testing,  we  believe  the 
interface  will  meet  FS  requirements  and  expectations.  Once  it  is  available,  we  will  attempt  to 
recruit  them  to  evaluate  the  interface  and  the  accompanying  information  on  fatigue 
countenneasures  medications. 

It  is  the  investigators’  opinion  that  the  web-based  F-PAS  will  eventually  replace  FAST™.  It  has 
three  unique  interfaces  for  shift  work  scheduling,  mishap  investigation,  and  mission  planning 
whereas  FAST™  has  but  one  interface  with  options  for  data  entry.  The  F-PAS  interfaces  are 
based  on  T A  of  the  various  AF  user  groups.  Cunently,  F-PAS  can  be  run  on  a  stand-alone 
computer  by  configuring  it  as  a  server,  but  with  addition  programming  effort  F-PAS  could  be 
converted  to  a  standard  Windows  application.  Further,  it  could  also  be  configured  to  run  on  a 
PDA,  but  would  also  require  redesign  of  the  displays  for  the  small  screen.  These  various  topics 
and  others  are  discussed  in  the  contract  final  report  (Eddy,  Miller,  &  Moise,  2008). 
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CONCLUSIONS 


The  F-PAS  mission  scheduler's  interface  should  provide  a  mechanism  that  allows  schedulers, 
aircrew,  and  FS  to  make  informed  decisions  with  respect  to  the  impact  of  the  mission  schedules 
on  fatigue  and  the  likelihood  of  performance  errors.  It  can  also  make  “what-if  ’  comparisons 
across  various  alternative  schedules.  Using  the  SAFTE™  model  in  F-PAS,  a  user  can  compare 
critical  events  (take-off,  aerial  refueling,  landing)  within  two  or  more  different,  fatiguing 
schedules.  The  model  may  identify,  for  example,  the  likelihood  that  an  individual  beginning  a 
flight  leg  has  not  fully  recovered  from  previous  flight  legs  and  is,  therefore,  at  an  elevated  risk  of 
causing  a  fatigue-induced  mishap  during  take-off  or  departure.  The  interface  can  suggest 
countenneasures  to  use  when  mishap  risk  is  higher  than  nonnal.  Aircrew  and  FS  should  use  the 
tool’s  prediction  of  performance  to  determine  the  best  course  of  action  regarding  possible  Go-No 
Go  pill  administration. 

In  usability  testing,  the  input  component  was  found  to  be  both  time  efficient  and  provide 
extensive  optional  guidance  with  respect  to  avoiding  or  minimizing  fatigue.  The  output 
component  of  the  interface  was  shown  to  identify  clearly  the  “fatigue  points”  in  the  proposed 
schedule,  based  upon  SAFTE™  calculations.  The  output,  once  extended,  will  allow  side-by-side 
comparison  of  several  candidate  schedules.  When  schedule  changes  or  alternative  options  for 
sleep  have  been  tried  and  found  insufficient  for  fatigue  risk  management,  phannaceutical 
countenneasures  can  be  investigated  within  the  modeling  tool.  These  should  be  understood 
easily  by  aircrew  who  are  not  physicians.  F-PAS  has  the  potential  to  replace  the  research- 
oriented  FAST™ 
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APPENDIX  A:  F/PAS  MISSION  SCHEDULER  INTERFACE  USABILITY  QUESTIONNAIRE 

NTI,  Inc.,  September  2008 


Years  of  active  duty: _  FAST  user?  Yes  No  (circle  one) 

Please  rate  the  ease  of  application  of  the  interface  to  the  intended  task:  the  simplicity  with  which  the 
interface  can  be  employed  to  determine  whether  fatigue  was  a  factor  in  a  mishap.  In  an  ideal  world,  the 
interface  would  be  totally  natural  and  predictable  in  behavior.  Nothing  should  obstruct  your  progress  in 
completing  this  task. 

•  Very  acceptable  (1) 

•  Acceptable  (2) 

•  Borderline  (3) 

•  Unacceptable  (4) 

•  Very  unacceptable  (5) 

Rate  the  performance  of  the  interface:  the  speed  with  which  the  interface  responds  to  requests. 

•  Very  acceptable  (1) 

•  Acceptable  (2) 

•  Borderline  (3) 

•  Unacceptable  (4) 

•  Very  unacceptable  (5) 

Rate  the  support  information  for  the  interface:  the  information  available  to  acquire,  use  and  support  the 
interface.  Encompasses  initial  instructions,  user  guides,  tutorials,  integrated  assistance,  context  sensitive 
help. 

•  Very  acceptable  (1) 

•  Acceptable  (2) 

•  Borderline  (3) 

•  Unacceptable  (4) 

•  Very  unacceptable  (5) 

Rate  the  interface  s  function:  the  overall  capabilities  of  the  interface. 

•  Very  acceptable  (1) 

•  Acceptable  (2) 

•  Borderline  (3) 

•  Unacceptable  (4) 

•  Very  unacceptable  (5) 

Please  discuss  with  the  observer: 

•  What  were  your  objectives  as  you  tested  this  interface? 

•  Was  the  scope  of  the  usability  testing  that  you  did  adequate  to  meet  your  objectives? 

•  Can  you  suggest  another  method  of  raw  data  entry  that  would  reduce  time,  prevent  entry  errors, 
and  provide  greater  awareness  of  data  conflicts/errors? 

•  Can  you  suggest  other  data  editing  methods  that  would  work  on  a  web  page  and  would  be  more 
powerful  for  making  changes? 

•  Could  the  interface  graph  be  formatted  differently  to  better  assist  you  in  completing  your  mishap 
investigation  and  report? 

•  What  other  improvements  should  be  made  to  the  interface? 
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APPENDIX  B:  NTI  F-PAS  MISSION  SCHEDULER  TOOL  USABILITY  DATA  COLLECTION  FORM 


Data: 

On  a  separate  page,  keep  orderly,  transcribable  notes  of  the  pathways  the  participants  take,  problems  participants  have  and  what  participants  say  as 
they  work.  Definitions  for  the  table,  below: 

•  Number  of  subtask  assists:  When  the  participant  cannot  proceed  on  a  subtask,  the  observer  gives  direct  procedural  help  to  allow  the  test  to 
proceed. 

•  Number  of  subtask  errors:  Instances  where  test  participant  had  to  attempt  portions  of  the  task  more  than  once. 

•  Number  of  subtask  reversals:  Number  of  times  participant  had  to  “back  up”  to  find  something  on  a  previous  page  that  they  needed  on  the 
current  page. 

•  Subtask  completion  (Y/N):  Yes  =  complete  and  correct  achievement  of  subtask  goal. 

•  Problem  severity  (0/1/2):  0  =  no  problem;  1  =minor  (users  are  annoyed,  but  this  does  not  keep  them  from  completing  the  scenario);  2  =  show 
stopper  (if  we  don't  fix  this,  users  will  not  be  able  to  complete  the  scenario;  and/or  many  users  will  be  frustrated,  and  they  may  give  up). 


Subtask 

#  Assists 

#  Errors 

#  Reversals 

Severity 

Completion 

1.  Start  time  at  Main  Menu: 

2.  Choose  “Mission  Scheduler — New  Schedule.”  Success  =  appearance  of  “Edit  schedule” 
screen. 

0  1  2 

Yes  No 

3.  Choose  “Edit  Properties.”  Success  =  appearance  “Edit  Properties”  pop-up  window. 

0  1  2 

Yes  No 

4.  Enter  schedule  name,  starting  date,  and  starting  time  (calendar),  and  choose  Zulu  time 
(Zulu). 

0  1  2 

Yes  No 

5.  Enter  Starting  location  (Travis)  without  DST.  Success  =  proper  starting  location  displayed 
on  “Edit  Properties”  pop-up. 

0  1  2 

Yes  No 

6.  Click  “Save  and  Exit.”  Success  =  appearance  of  “Edit  Schedule”  screen. 

0  1  2 

Yes  No 

7.  Choose  “Edit  Schedule.”  Success  =  appearance  “Visual  Schedule  Editor”  pop-up  window. 
PROBLEM:  The  instruction  here  is  to  choose  “Edit  Properties.”  Needs  second  paragraph 
cueing  the  selection  of  schedule  editor  after  properties  have  been  entered. 

0  1  2 

Yes  No 

8.  Enter  first  work  period,  Travis  (via  Randolph)  to  Pope.  PROBLEM:  No  instruction  given 
on  the  screen  to  double-click  at  the  desired  time  in  the  calendar  to  enter  a  work  period. 
Success  =  appearance  of  proper  work  period  on  calendar  on  “Visual  Schedule  Editor”  pop¬ 
up  window.  NOTE:  Visual  Schedule  Editor  page  needs  a  save-without-exit  option. 

NOTE:  Hover  window  over  work  period  shows  time  in  12-h  clock  when  time  base  is  Zulu 

0  1  2 

Yes  No 
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Subtask 

#  Assists 

#  Errors 

#  Reversals 

Severity 

Completion 

and  24-h  clock  should  be  shown. 

9.  Enter  second  work  period,  Pope  to  Lajes.  Success  =  appearance  of  proper  work  period  on 
calendar  on  “Visual  Schedule  Editor”  pop-up  window. 

0  1  2 

Yes  No 

10.  Enter  third  work  period,  Lajes  to  Ramstein.  Success  =  appearance  of  proper  work  period 
on  calendar  on  “Visual  Schedule  Editor”  pop-up  window.  NOTE:  To  this  point, 
incredibly  slow  response  times  from  novasciinc.com. 

0  1  2 

Yes  No 

11.  Enter  first  Event,  take-off  from  Travis.  Success  =  appearance  of  proper  event  on  calendar 
on  “Visual  Schedule  Editor”  pop-up  window.  PROBLEM:  Again,  no  cue  to  double-click 
in  calendar.  SHOW-STOPPER:  First  work  period's  start  time  has  mysteriously  reverted 
to  midnight  of  first  day!  The  edit  function  failed  to  fix  the  problem.  PROBLEM:  No 
instruction  on  page  about  how  to  edit  a  work  period.  PROBLEM:  Must  clcik  on  empty, 
non-work  space  in  calendar  to  insert  an  event,  even  if  the  event  occurs  during  a  work 
period.  PROBLEM:  Pop-up  window  for  selection  of  Work,  Sleep  or  Event  is  displayed 
during  Event  save. 

0  1  2 

Yes  No 

12.  Enter  second  Event,  Landing  at  Randolph.  Success  =  appearance  of  proper  event  on 
calendar  on  “Visual  Schedule  Editor”  pop-up  window.  CEIANGE:  In  the  add-an-event 
window,  place  the  “Start  Time”  filed  above  the  “Event  Type”  field  for  better  sequencing  of 
user  inputs. 

0  1  2 

Yes  No 

13.  Enter  third  Event,  take-off  from  Randolph.  Success  =  appearance  of  proper  event  on 
calendar  on  “Visual  Schedule  Editor”  pop-up  window. 

0  1  2 

Yes  No 

14.  Enter  fourth  Event,  landing  at  Pope.  Success  =  appearance  of  proper  event  on  calendar  on 
“Visual  Schedule  Editor”  pop-up  window. 

0  1  2 

Yes  No 

15.  Enter  fifth  Event,  take-off  from  Pope.  Success  =  appearance  of  proper  event  on  calendar 
on  “Visual  Schedule  Editor”  pop-up  window. 

0  1  2 

Yes  No 

16.  Enter  sixth  Event,  landing  at  Lajes.  Success  =  appearance  of  proper  event  on  calendar  on 
“Visual  Schedule  Editor”  pop-up  window. 

0  1  2 

Yes  No 

17.  Enter  seventh  Event,  take-off  from  Lajes.  Success  =  appearance  of  proper  event  on 
calendar  on  “Visual  Schedule  Editor”  pop-up  window. 

0  1  2 

Yes  No 

18.  Enter  eighth  Event,  landing  at  Ramstein.  Success  =  appearance  of  proper  event  on  calendar 
on  “Visual  Schedule  Editor”  pop-up  window. 

0  1  2 

Yes  No 

19.  Enter  first  crew  rest  sleep  period  at  Pope.  Success  =  appearance  of  proper  sleep  period  on 

0  1  2 

Yes  No 
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Subtask 

#  Assists 

#  Errors 

#  Reversals 

Severity 

Completion 

calendar  on  “Visual  Schedule  Editor”  pop-up  window.  PROBLEM:  What  was  sleep 
before  the  schedule  started? 

20.  Enter  second  crew  rest  sleep  period  at  Lajes.  Success  =  appearance  of  proper  sleep  period 
on  calendar  on  “Visual  Schedule  Editor”  pop-up  window. 

0  1  2 

Yes  No 

21.  Save  and  exit  “Visual  Schedule  Editor”  pop-up  window.  Success  =  appearance  of  “Edit 
Schedule”  screen. 

0  1  2 

Yes  No 

22.  Save  the  schedule.  Success  =  message  confirming  saving  of  schedule. 

0  1  2 

Yes  No 

23.  Display  the  effectiveness  graph.  Success  =  appearance  of  graph. 

RECOMMENDATION:  Use  1-hour  resolution  as  default;  easier  to  get  the  big  picture 
with  1 -hour  than  with  15 -min  res. 

0  1  2 

Yes  No 

24.  Switch  to  1  -hour  resolution.  Success  =  re-display  of  graph. 

0  1  2 

Yes  No 

25.  End  time  for  data  entry: 

26.  Start  time  for  fatigue  countermeasure  applications: 

27. 

0  1  2 

Yes  No 

28. 

0  1  2 

Yes  No 

29. 

0  1  2 

Yes  No 

30. 

0  1  2 

Yes  No 

31. 

0  1  2 

Yes  No 

32. 

0  1  2 

Yes  No 

33. 

0  1  2 

Yes  No 

34. 

0  1  2 

Yes  No 

35. 

0  1  2 

Yes  No 

36. 

0  1  2 

Yes  No 

37. 

0  1  2 

Yes  No 

38. 

0  1  2 

Yes  No 

39. 

0  1  2 

Yes  No 
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Subtask 
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#  Assists 

#  Errors 

#  Reversals 

Severity 

Completion 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

0  1  2 

Yes  No 

ted,  Public  Affairs  Case  File  No.  09-402, 
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Subtask 


#  Assists  #  Errors  #  Reversals  Severity  Completion 

0  12  Yes  No 

0  12  Yes  No 

0  12  Yes  No 

0  12  Yes  No 


71.  End  time: 


NA  NA 


Comments: 
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APPENDIX  C:  MISSION  SCHEDULER  SCENARIO 

Usability  Test,  25  Sep  2008 

DATA 


Mobility  Mission  no.  456 

Travis  AFB,  CA,  to  Ramstein  AB,  Germany 
Show  time:  Take-off  minus  3  hrs 
Call  sign:  Mickey21 
Home  base,  Travis  AFB,  CA 

Pre-mission  sleep:  predicted  sleep  period  0600Z-1200Z,  predicted  sleep  quality  “excellent” 

1st  Work  Period  (13.5  hrs) 

11.  Show  time,  Travis  AFB:  23APR1348Z 

12.  Take-off,  Travis  AFB:  23APR1648Z 

13.  Landing  Randolph  AFB,  TX:  23APR2000Z  (3.2  hrs  enroute) 

9.  Take-off,  Randolph  AFB,  TX:  23APR2350Z 

10.  Landing,  Pope  AFB,  NC:  24APR0320Z  (3.5  hrs  enroute) 

1 1 .  Debrief,  Pope  AFB,  NC:  1  hour 

Crew  rest:  18  hrs,  predicted  sleep  period  0830Z-1330Z,  predicted  sleep  quality  “fair” 

2nd  Work  Period  (9  hrs) 

•  Showtime,  Pope  AFB,  NC:  24APR2115Z 

•  Take-off,  Pope  AFB,  NC:  25APR0015Z 

•  Landing,  Lajes  AB,  Azores:  25APR0520Z  (5  hrs  enroute) 

•  Debrief,  Lajes  AB,  Azores:  1  hour 

Crew  rest:  13  hrs,  predicted  sleep  period  0700Z-1300Z,  predicted  sleep  quality  “fair” 

3rd  Work  Period  (9.5  hrs) 

•  Show  time,  Lajes  AB,  Azores:  25APR1830Z 

•  Take-off,  Lajes  AB,  Azores:  25APR2130Z 

•  Landing,  Ramstein  AB,  Germany:  26APR0300Z  (5.5  hrs  enroute) 

•  Debrief,  Ramstein  AB,  Germany:  1  hour 
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Mission  Scheduler  Scenario 

Usability  Test,  25  Sep  2008 

Test  Instructions  (Simulated  Help  File) 

Data  Entry: 

•  From  the  Main  menu,  choose  “Mission  Scheduler — New  Schedule”  and  then  “Edit 
Properties.” 

•  Referring  to  the  test  scenario,  enter  the  schedule  properties  as  instructed.  Use  Zulu  time 
and  do  not  use  Daylight  Savings  Time  for  this  whole  test. 

•  Return  to  the  “Edit  Schedule”  screen.  Choose  “Edit  Schedule.” 

•  Referring  to  the  test  scenario,  enter  each  of  the  three  work  periods,  from  show  time 
through  the  end  of  the  mission  debrief. 

•  Referring  to  the  test  scenario,  enter  the  four  pairs  of  take-offs  and  landing  as  events, 
being  sure  to  enter  the  location. 

•  Referring  to  the  test  scenario,  enter  the  three  predicted  sleep  periods  (automated 
prediction  is  not  yet  available  in  the  software);  assign  the  correct  sleep  quality  to  each 
one. 

•  Save  and  exit  the  “Visual  Schedule  Editor”  pop-up  window. 

•  Save  the  schedule. 

•  Display  the  graph,  and  change  to  1-hour  resolution. 

•  Add  pharmacological  manipulations  for  future  testing. 
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